The extra space is being returned by Vimeo themselves - it's the height of the iframe which contains the embedded content. See the following:
https://vimeo.com/api/oembed.json?url=https://vimeo.com/35686630
The response is:
{
"type": "video",
"version": "1.0",
"provider_name": "Vimeo",
"provider_url": "https:\/\/vimeo.com\/",
"title": "Stv-Visuals 21:9 Free Loops",
"author_name": "Stv-Visuals",
"author_url": "https:\/\/vimeo.com\/stvvisuals",
"is_plus": "0",
"account_type": "basic",
"html": "<iframe src=\"https:\/\/player.vimeo.com\/video\/35686630?h=cc53039083&app_id=122963\" width=\"480\" height=\"204\" frameborder=\"0\" allow=\"autoplay; fullscreen; picture-in-picture\" allowfullscreen title=\"Stv-Visuals 21:9 Free Loops\"><\/iframe>",
"width": 480,
"height": 204,
"duration": 25,
"description": "http:\/\/www.facebook.com\/Stvvisuals\n\nLike our page and download the loops for free!\nStv-Visuals 21:9 Free Loops",
"thumbnail_url": "https:\/\/i.vimeocdn.com\/video\/244200203-99a5d4e8b7ef0a064558ec3f14a32c252135e5d02fa8fd633f4ba11565bb125e-d_295x166",
"thumbnail_width": 295,
"thumbnail_height": 124,
"thumbnail_url_with_play_button": "https:\/\/i.vimeocdn.com\/filter\/overlay?src0=https%3A%2F%2Fi.vimeocdn.com%2Fvideo%2F244200203-99a5d4e8b7ef0a064558ec3f14a32c252135e5d02fa8fd633f4ba11565bb125e-d_295x166&src1=http%3A%2F%2Ff.vimeocdn.com%2Fp%2Fimages%2Fcrawler_play.png",
"upload_date": "2012-01-26 07:36:40",
"video_id": 35686630,
"uri": "\/videos\/35686630"
}
The 204 height of the iframe is double the height of the actual video, so you get 51 pixels of space either side of the video within the iframe.