YouTube Data API (v3): Introduction With Spring Boot
YouTube is a video-sharing website where users can view and upload videos. For both web and app development, sometimes we need to access information from YouTube videos.
You may also like: Build a Spring Boot API Using Spring Data JPA and Hibernate
In this case, we have to use the API's provided by Google developers. The main goal of this article is to introduce how we can use YoutTube APIs with Java. Here, we will use Spring Boot with Hibernate JPA and libraries from Google to implement the example.
Pre-Conditions
To access the YouTube Data API, we have to follow some basic instructions, like create a Google account, a project, credentials, etc. Maybe, it would be better to follow these instructions listed here . If we follow the steps from this link, we first need the ClientId, API key, client secret, etc.
Next, we can start our implementation. We need to put our credentials into the application.properties file. To run the GitHub example (link is given below), we also need to set up Java, MySQL, and Gradle.
API Docs
In this article, we are mainly concerned with video information. A YouTube video property mainly contains a channel, duration, description, and statistics (likes, dislikes, etc.). These properties are available in different resources of the YouYube API, such as video, channel, activity, etc. These resources can also perform list, insert, update, and delete operations.
The last part of this tutorial will be to customize the YouYube API so that we can define what we want. Just like the statistics part only contains the number of likes, dislikes, pageviews, etc., the ContentDetails part only contains video definitions, captions, country restrictions, etc. — and all of these components contain nested objects. Combined, we can get all the properties of a video, but here, we will be focusing on important parts only.
Implementation
With that said, we will be implementing the YouTube Data API in a bit of different way. We will use a REST API to save extracted data to the database. For database entities, we will create a corresponding JPA entity. We have divided most of the video properties depending on a different part. Entities are:
- YoutubeChannelInfo
- YouTubeVideoInfo
- YoutubeVideoStatistics
Additionally, we will have a BaseEntity for common properties, and another entity — CrawlingInfo — to keep track of crawled data.
We will collect the following data for the channel information:
@Entity(name = "youtube_channel")
@Data
@EqualsAndHashCode(callSuper=true)
public class YoutubeChannelInfo extends BaseEntity{
@Column(name = "channel_id")
private String channelId;
@Column(name = "name")
private String name;
@Column(name = "subscription_count")
private long subscriptionCount;
}For video statistics, we will collect the following data:
@Entity(name = "youtube_video_stat")
@Data
@EqualsAndHashCode(callSuper=true)
public class YoutubeVideoStatistics extends BaseEntity{
@Column(name = "like_count")
private long likeCount;
@Column(name = "dislike_count")
private long dislikeCount;
@Column(name = "view_count")
private long viewCount;
@Column(name = "favourite_count")
private long favouriteCount;
@Column(name = "comment_count")
private long commentCount;
@Column(name = "video_id")
private String videoId;
}And for common video properties, we will collect the following data:
@Entity(name = "youtube_video_info")
@Data
@EqualsAndHashCode(callSuper=true)
public class YouTubeVideoInfo extends BaseEntity{
@Column(name = "video_id")
private String videoId;
@Column(name = "title")
private String title;
@Column(name = "thumbnail_url")
private String thumbnailUrl;
@Column(name = "description")
private String description;
@Column(name = "published_date")
private Date publishedDate;
@Column(name = "definition")
private String videoDefinition;
@Column(name = "duration")
private String videoDuration;
@Column(name = "caption")
private String videoCaption;
@Column(name = "projection")
private String videoprojection;
@Column(name = "country_restricted")
private String countryRestricted;
@Column(name = "keyword")
private String keyword;
@OneToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "channel_id",referencedColumnName = "channel_id")
private YoutubeChannelInfo channelInfo;
@OneToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "video_stat_id",referencedColumnName = "video_id")
private YoutubeVideoStatistics videoStatistics;
}Now, we will fetch video information through the YouTube Data API. So, we need some libraries provided by the Google team.
compile group: 'com.google.api-client', name: 'google-api-client', version: '1.25.0'
compile group: 'com.google.apis', name: 'google-api-services-youtube', version: 'v3-rev212-1.25.0'
compile group: 'com.google.apis', name: 'google-api-services-youtubeAnalytics', version: 'v1-rev14-1.13.2-beta'
compile group: 'com.google.apis', name: 'google-api-services-youtubereporting', version: 'v1-rev1-1.20.0'
compile("com.google.oauth-client:google-oauth-client-jetty:1.25.0") {
exclude group: 'org.mortbay.jetty', module: 'servlet-api'
}
compile group: 'com.google.http-client', name: 'google-http-client-jackson2', version: '1.25.0'
compile group: 'com.google.collections', name: 'google-collections', version: '1.0'Other libraries are also used for Spring Boot and database support. We can find these libraries at gradle file .
At first, we have to initialize YouTube (v3) properties which supports core YouTube features such as uploading videos, creating and managing playlists, searching for content, and much more. But here, we will first implement some basic properties.
youtube = new YouTube.Builder(HTTP_TRANSPORT, JSON_FACTORY, new HttpRequestInitializer() {
public void initialize(HttpRequest request) throws IOException {
}
}).setApplicationName("YoutubeVideoInfo")
.setYouTubeRequestInitializer
(new YouTubeRequestInitializer(env.getProperty("youtube.apikey"))).build();
When I was creating my credentials, I had to create an application name YouTubeVideoInfo that is assigned as application name. The property youtube.apikey is located in the application.properties file. We have to replace youtube.apikey's value with our ones.
Now, we will fetch some information using this. Initially, we use id, snippet to fetch some basic data.
YouTube.Search.List search = youtube.search().list("id,snippet");
//search keyword eg. "IELTS"
search.setQ(queryTerm);
//return only video type data
search.setType("video");
// number of video data return. maximum 50
search.setMaxResults(NUMBER_OF_VIDEOS_RETURNED);Here, queryTerm will be our search key, and video will be the data type so that it will return video type data information. In a youtube API list request, maximum 50 videos information returns as response.
SearchListResponse searchResponse = search.execute();
List<SearchResult> searchResultList = searchResponse.getItems();
if (searchResultList != null) {
extractAndSave(searchResultList.iterator(), queryTerm);
}
crawlingInfo.setNextPageToken(searchResponse.getNextPageToken());
crawlingInfoService.update(crawlingInfo);
System.out.println("Next Page token : " + searchResponse.getNextPageToken());Here, the search.execute() method will return the expected result. So, we have to parse this response. And another thing: CrawlingInfo class will be used to maintain the next and previous page. The YouTube API returns a maximum of 50 pieces of data in a request with the next page token. If we want to access next page we have to add nextPageToken at pageToken properties on request, then it will move to the next page. That's why we are keeping this track using crawlingInfo entity.
We will use more queries on the remaining code to extract channel information, content details, statistics, etc. for every video and then save to the database.
public List<Object> getYoutubeVideoList(String queryTerm,long pageToCrawl) {
try {
youtube = new YouTube.Builder(HTTP_TRANSPORT, JSON_FACTORY, new HttpRequestInitializer() {
public void initialize(HttpRequest request) throws IOException {
}
}).setApplicationName("YoutubeVideoInfo")
.setYouTubeRequestInitializer(new YouTubeRequestInitializer(env.getProperty("youtube.apikey"))).build();
YouTube.Search.List search = youtube.search().list("id,snippet");
search.setQ(queryTerm);
search.setType("video");
search.setMaxResults(NUMBER_OF_VIDEOS_RETURNED);
for (int i = 0; i < pageToCrawl; i++) {
String pageToken = null;
CrawlingInfo crawlingInfo = crawlingInfoService.getBySearchKey(queryTerm);
if (crawlingInfo != null && crawlingInfo.getNextPageToken() != null) {
pageToken = crawlingInfo.getNextPageToken();
count = crawlingInfo.getTotalCount();
crawlingInfo.setCurrentPageToken(pageToken);
} else if (crawlingInfo == null) {
crawlingInfo = new CrawlingInfo();
count = 0;
crawlingInfo.setSearchKey(queryTerm);
crawlingInfo.setCurrentPageToken(null);
}
if (pageToken != null) {
search.setPageToken(pageToken);
}
SearchListResponse searchResponse = search.execute();
List<SearchResult> searchResultList = searchResponse.getItems();
if (searchResultList != null) {
extractAndSave(searchResultList.iterator(), queryTerm);
}
crawlingInfo.setNextPageToken(searchResponse.getNextPageToken());
crawlingInfoService.update(crawlingInfo);
System.out.println("Next Page token : " + searchResponse.getNextPageToken());
}
} catch (GoogleJsonResponseException e) {
System.err.println("There was a service error: " + e.getDetails().getCode() + " : "
+ e.getDetails().getMessage());
} catch (IOException e) {
System.err.println("There was an IO error: " + e.getCause() + " : " + e.getMessage());
} catch (Throwable t) {
t.printStackTrace();
}
return null;
}
private void extractAndSave(Iterator<SearchResult> iteratorSearchResults, String query) throws IOException {
if (!iteratorSearchResults.hasNext()) {
System.out.println(" There aren't any results for your query.");
}
while (iteratorSearchResults.hasNext()) {
SearchResult singleVideo = iteratorSearchResults.next();
System.out.println("Video number = " + count + " Inserting video Information " + singleVideo.getId().getVideoId());
count++;
YouTubeVideoInfo youTubeVideoInfo = youtubeVideoInfoService.getByVideoId(singleVideo.getId().getVideoId());
if (youTubeVideoInfo == null) {
youTubeVideoInfo = new YouTubeVideoInfo();
ResourceId rId = singleVideo.getId();
youTubeVideoInfo.setKeyword(query);
youTubeVideoInfo.setDescription(singleVideo.getSnippet().getDescription());
youTubeVideoInfo.setPublishedDate(new Date(singleVideo.getSnippet().getPublishedAt().getValue()));
if (rId.getKind().equals("youtube#video")) {
Thumbnail thumbnail = singleVideo.getSnippet().getThumbnails().getDefault();
youTubeVideoInfo.setVideoId(rId.getVideoId());
youTubeVideoInfo.setTitle(singleVideo.getSnippet().getTitle());
youTubeVideoInfo.setThumbnailUrl(thumbnail.getUrl());
youTubeVideoInfo.setChannelInfo(getChannelDetailsById(singleVideo.getSnippet().getChannelId()));
youTubeVideoInfo.setVideoStatistics(getVideosStatistics(rId.getVideoId()));
}
youtubeVideoInfoService.save(youTubeVideoInfo);
} else {
System.out.println("Video Already exists... ");
}
}
}
private YoutubeChannelInfo getChannelDetailsById(String channelId) throws IOException {
YouTube.Channels.List channels = youtube.channels().list("snippet, statistics");
YoutubeChannelInfo youtubeChannelInfo = new YoutubeChannelInfo();
youtubeChannelInfo.setChannelId(channelId);
channels.setId(channelId);
ChannelListResponse channelResponse = channels.execute();
Channel c = channelResponse.getItems().get(0);
youtubeChannelInfo.setName(c.getSnippet().getTitle());
youtubeChannelInfo.setSubscriptionCount(c.getStatistics().getSubscriberCount().longValue());
YoutubeChannelInfo channelInfo = youtubeChannelService.getByChannelId(channelId);
if (channelInfo == null) {
youtubeChannelService.save(youtubeChannelInfo);
} else {
return channelInfo;
}
return youtubeChannelInfo;
}
public YoutubeVideoStatistics getVideosStatistics(String id) throws IOException {
YouTube.Videos.List list = youtube.videos().list("statistics");
list.setId(id);
Video v = list.execute().getItems().get(0);
YoutubeVideoStatistics statistics = new YoutubeVideoStatistics();
statistics.setVideoId(id);
statistics.setLikeCount(v.getStatistics().getLikeCount() !=null ? v.getStatistics().getLikeCount().longValue():0);
statistics.setDislikeCount(v.getStatistics().getDislikeCount() != null ? v.getStatistics().getDislikeCount().longValue():0);
statistics.setFavouriteCount(v.getStatistics().getFavoriteCount() != null ? v.getStatistics().getFavoriteCount().longValue():0);
statistics.setCommentCount(v.getStatistics().getCommentCount() != null ? v.getStatistics().getCommentCount().longValue() : 0);
statistics.setViewCount(v.getStatistics().getViewCount() != null ? v.getStatistics().getViewCount().longValue() : 0);
youtubeVideoStatService.save(statistics);
return statistics;
}
public YouTubeVideoInfo getCoontentDetails(String id, YouTubeVideoInfo youTubeVideoInfo) throws IOException {
YouTube.Videos.List list = youtube.videos().list("contentDetails");
list.setId(id);
Video v = list.execute().getItems().get(0);
youTubeVideoInfo.setVideoDefinition(v.getContentDetails().getDefinition());
youTubeVideoInfo.setVideoCaption(v.getContentDetails().getCaption());
youTubeVideoInfo.setVideoprojection(v.getContentDetails().getProjection());
youTubeVideoInfo.setCountryRestricted(v.getContentDetails().getCountryRestriction().toPrettyString());
return youTubeVideoInfo;
}If we take a look on the previous code snippet, we can see that, for statistics and contentDetails, we are using videos resources and list operation. And for channel information, we are using channel resources and list operation. Here, I have represented all list operations of the YouTube Data API. And with similar approach, we can use other operations smoothly.
The code examples from this article are available here.
Further Reading
Build a Spring Boot API Using Spring Data JPA and Hibernate
