The Category Facet uses assetVocabularyCategoryIds
field for aggregation. The required mapping and indexer changes have been available since 7.4 GA46/U46 (LPS-156503: Display, Scope and Filter Categories by Vocabulary in the Category Facet).
Note: Reindex is required after all upgrades.