{"id":1364,"date":"2022-08-17T05:50:31","date_gmt":"2022-08-17T05:50:31","guid":{"rendered":"https:\/\/www.webharvy.com\/blog\/?p=1364"},"modified":"2022-09-16T05:50:55","modified_gmt":"2022-09-16T05:50:55","slug":"web-scraping-youtube-video-details","status":"publish","type":"post","link":"https:\/\/www.webharvy.com\/blog\/web-scraping-youtube-video-details\/","title":{"rendered":"Web Scraping YouTube Video Details"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/www.webharvy.com\/articles\/what-is-web-scraping.html\">Web Scraping<\/a> is the process of automatically extracting data from websites using software known as <a href=\"https:\/\/www.webharvy.com\/articles\/best-web-scraping-software.html\">web scrapers<\/a>. In this article we will learn how web scraping can be used to extract YouTube video details. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Scraping YouTube<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">YouTube Video Data can be scraped from :<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Video recommendations (home) page of an account<\/li><li>YouTube video search results page<\/li><li>Videos page of a YouTube channel <\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">The following data related to a video can be scraped:<\/p>\n\n\n\n<ol class=\"wp-block-list\"><li>Video title<\/li><li>Video URL<\/li><li>Thumbnail<\/li><li>Views<\/li><li>Likes<\/li><li>Channel which posted the video<\/li><li>Number of subscribers which the channel has<\/li><li>etc. <\/li><\/ol>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Software to use<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/www.webharvy.com\/index.html\">WebHarvy <\/a>is a visual web scraping software which can be used to scrape data from any website including YouTube. WebHarvy can also scrape data from various other Google services like <a href=\"https:\/\/www.webharvy.com\/articles\/scraping-google-maps.html\">Maps<\/a>, <a href=\"https:\/\/www.webharvy.com\/articles\/scraping-google-shopping.html\">Shopping<\/a>, Trends, <a href=\"https:\/\/www.webharvy.com\/blog\/how-to-scrape-google-jobs-scraping-job-details\/\">Jobs <\/a>etc. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Scraping YouTube Search Results<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The video displayed below shows how video data can be scraped from YouTube search results (video listings).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/gist.github.com\/sysnucleus\/a020ba1f89d489d1f045f36866e9cbf8\">The codes used in the video can be found here<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Web Scraping YouTube Video Data | WebHarvy 2022\" width=\"1200\" height=\"675\" src=\"https:\/\/www.youtube.com\/embed\/PhFNoE-ha7s?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Scraping any YouTube channel&#8217;s video data<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The video shown below demonstrates how video details can be scraped from any YouTube channel. The <a href=\"https:\/\/gist.github.com\/sysnucleus\/639529145b4a8dd06be2c75d479eaeec\">code used  for pagination can be found here<\/a>.<\/p>\n\n\n\n<figure class=\"wp-block-embed is-type-video is-provider-youtube wp-block-embed-youtube wp-embed-aspect-16-9 wp-has-aspect-ratio\"><div class=\"wp-block-embed__wrapper\">\n<iframe loading=\"lazy\" title=\"Web Scraping YouTube Video Details from any Channel | WebHarvy\" width=\"1200\" height=\"675\" src=\"https:\/\/www.youtube.com\/embed\/WiNTe_UfJQs?feature=oembed\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen><\/iframe>\n<\/div><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Steps to follow to scrape YouTube Video Data<\/h2>\n\n\n\n<ul class=\"wp-block-list\"><li>The first step is to <a href=\"https:\/\/www.webharvy.com\/download.html\">download <\/a>and install WebHarvy in your computer. <\/li><li>WebHarvy has a browser like user interface within which you can load and navigate web pages<\/li><li>Open YouTube website within WebHarvy and navigate to the video listings page from which you need to scrape data<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"550\" src=\"https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-2-1024x550.png\" alt=\"YouTube Videos Loaded in WebHarvy\" class=\"wp-image-1367\" srcset=\"https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-2-1024x550.png 1024w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-2-300x161.png 300w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-2-768x413.png 768w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-2-1536x826.png 1536w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-2.png 1600w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\"><li>Click the <strong>Start <\/strong>button under <strong>Configuration <\/strong>pane in the <strong>Home <\/strong>menu (<a href=\"https:\/\/www.webharvy.com\/tour.html\">Start Configuration<\/a>)<\/li><li>Now we can start selecting the data which we need to scrape from the page. To select any data item displayed on the page, click over it. WebHarvy will display a <a href=\"https:\/\/www.webharvy.com\/tour1.html\">Capture window with various options<\/a>.<\/li><li>Before selecting data from videos listed on the page, we need to <a href=\"https:\/\/www.webharvy.com\/tour3.html\">configure pagination<\/a><\/li><li>YouTube loads more data (videos) on the same page as we scroll down. This is called infinite scroll or scroll to load pagination. To configure WebHarvy to automatically perform this action, go to the <strong>Configuration <\/strong>menu and click on the<strong> Set JavaScript<\/strong> button under <strong>Pagination <\/strong>pane. In the resulting window, paste the following JavaScript code. (<a href=\"https:\/\/www.webharvy.com\/tour3.html#JS\">Pagination using JavaScript<\/a>)<\/li><\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>els = document.getElementsByTagName('ytd-video-renderer');\nels&#91;els.length-1].scrollIntoView();<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\"><li>Now click anywhere on the page and select <strong>More Options &gt; Run Script <\/strong>from the resulting Capture window. Paste and apply the following code. (<a href=\"https:\/\/www.webharvy.com\/tour1.html#RunScript\">Running JavaScript code on page<\/a>)<\/li><\/ul>\n\n\n\n<pre class=\"wp-block-code\"><code>els = document.getElementsByTagName('ytd-video-renderer');\nparent = els&#91;0].parentElement;\nfor (var i = els.length - 1; i &gt;= 0; i--) {\n\tif(els&#91;i].parentElement !== parent) {\n\t\tparent.appendChild(els&#91;i]);\n\t}\n}<\/code><\/pre>\n\n\n\n<ul class=\"wp-block-list\"><li>Now we can start selecting data. <\/li><li>Click over the title of the first video. Select <strong><a href=\"https:\/\/www.webharvy.com\/tour1.html#ScrapeText\">Capture Text <\/a><\/strong>option from the resulting Capture window to select it for scraping. <\/li><li>To select the video URL use the <strong><a href=\"https:\/\/www.webharvy.com\/tour1.html#ScrapeURL\">Capture target URL<\/a> <\/strong>option<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-gallery has-nested-images columns-default is-cropped wp-block-gallery-1 is-layout-flex wp-block-gallery-is-layout-flex\">\n<figure class=\"wp-block-image size-full\"><img loading=\"lazy\" decoding=\"async\" width=\"1280\" height=\"720\" data-id=\"1368\" src=\"https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/selecting-text-url.gif\" alt=\"Selecting YouTube Video Details for Scraping\" class=\"wp-image-1368\"\/><\/figure>\n<\/figure>\n\n\n\n<ul class=\"wp-block-list\"><li>To follow the video link to select more video details use the <strong>Follow this link <\/strong>option after clicking on the video link\/title. (<a href=\"https:\/\/www.webharvy.com\/tour2.html\">Following links<\/a>)<\/li><li>Once the video details page is loaded, click anywhere on the page and select <strong>More Options &gt; Page &gt; Scroll down <\/strong>option. This is performed so that all data on page is loaded before we start selecting it for extraction. (<a href=\"https:\/\/www.webharvy.com\/tour1.html#ScrollPage\">Scrolling down page<\/a>)<\/li><li>Video details like views, likes, channel name, number of subscribers etc. can be selected from this page. <\/li><li>Once you have finished selecting all required data, click the <strong>Stop <\/strong>button under <strong>Configuration <\/strong>pane in <strong>Home <\/strong>menu. <\/li><li>You may now optionally <a href=\"https:\/\/www.webharvy.com\/tour4.html\">save the configuration<\/a><\/li><li>Click the <strong>Start Mine <\/strong>button to <a href=\"https:\/\/www.webharvy.com\/tour5.html\">start mining<\/a> data using the configuration which we just created.<\/li><li>The miner window&#8217;s data table will start to display data extracted from the page as mining progresses<\/li><\/ul>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"550\" src=\"https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-3-1024x550.png\" alt=\"Scraping YouTube Video Data\" class=\"wp-image-1369\" srcset=\"https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-3-1024x550.png 1024w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-3-300x161.png 300w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-3-768x413.png 768w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-3-1536x826.png 1536w, https:\/\/www.webharvy.com\/blog\/wp-content\/uploads\/2022\/08\/image-3.png 1600w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<ul class=\"wp-block-list\"><li>Once mining is finished you can <a href=\"https:\/\/www.webharvy.com\/tour6.html\">save the mined data to a file or to a database<\/a><\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Try WebHarvy<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You may download and try a free 15 day evaluation version of WebHarvy. To get started, <a href=\"https:\/\/www.webharvy.com\/articles\/getting-started.html\">please follow this link<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Have Questions?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">If you have any questions, please do not hesitate to <a href=\"https:\/\/www.webharvy.com\/support.html\">contact our technical support team<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Related<\/h2>\n\n\n\n<ol class=\"wp-block-list\"><li><a href=\"https:\/\/www.webharvy.com\/blog\/scraping-comments-from-youtube-videos\/\">Scraping YouTube Comments<\/a><\/li><\/ol>\n","protected":false},"excerpt":{"rendered":"<p>Web Scraping is the process of automatically extracting data from websites using software known as web scrapers. In this article we will learn how web scraping can be used to extract YouTube video details. Scraping YouTube YouTube Video Data can be scraped from : Video recommendations (home) page of an account YouTube video search results &#8230; <a title=\"Web Scraping YouTube Video Details\" class=\"read-more\" href=\"https:\/\/www.webharvy.com\/blog\/web-scraping-youtube-video-details\/\" aria-label=\"Read more about Web Scraping YouTube Video Details\">Read more<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,7,8],"tags":[175],"class_list":["post-1364","post","type-post","status-publish","format-standard","hentry","category-use-case","category-web-scraping-workshop","category-webharvy","tag-youtube"],"_links":{"self":[{"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/posts\/1364","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/comments?post=1364"}],"version-history":[{"count":8,"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/posts\/1364\/revisions"}],"predecessor-version":[{"id":1388,"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/posts\/1364\/revisions\/1388"}],"wp:attachment":[{"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/media?parent=1364"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/categories?post=1364"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.webharvy.com\/blog\/wp-json\/wp\/v2\/tags?post=1364"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}