Building player without assets in AssetBundleBuild[] built asset bundles

In my building pipeline, I use BuildPipeline.BuildAssetBundles(string outputPath, AssetBundleBuild[] builds, BuildAssetBundleOptions assetBundleOptions, BuildTarget targetPlatform). This way, I can determine my AssetBundle map by script without having to mess with the meta files in the project.

However, When I build a player after, It seems not to take into account the built AssetBundles and it builds everything in the player, even the assets included in an AssetBundle. It seems I need to mark my asset manually in the editor if I want them to be excluded from a build. Am I missing something? Right now, it seems this method is completely useless.