bots: Refactor Youtube bot.

This commit is contained in:
Sivagiri Visakan 2017-12-09 15:32:16 +05:30 committed by showell
parent f947ff44f8
commit 6f9d010ed3
18 changed files with 663 additions and 42 deletions

Binary file not shown.

After

Width:  |  Height:  |  Size: 80 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 29 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 32 KiB

View file

@ -0,0 +1,64 @@
# YouTube bot
The YouTube bot is a Zulip bot that can search for videos from [YouTube](https://www.youtube.com/).
To use the YouTube bot, you can simply call it with `@YouTube` followed
by a keyword(s), like so:
```
@YouTube funny cats
```
## Setup
Before starting you will need a Developer's API key to run the bot.
To obtain a API key, follow the following steps :
1. Create a project in the [Google Developers Console](https://console.developers.google.com/)
2. Open the [API Library](https://console.developers.google.com/apis/library?project=_)
in the Google Developers Console. If prompted, select a project or create a new one.
In the list of APIs, select `Youtube Data API v3` and make sure it is enabled .
3. Open the [Credentials](https://console.developers.google.com/apis/credentials?project=_) page.
4. In the Credentials page , select *Create Credentials > API key*
5. Open `zulip_bots/bots/youtube/youtube.conf` in an editor and
and change the value of the `key` attribute to the API key
you generated above.
6. And that's it ! See Configuration section on configuring the bot.
{!running-a-bot.md!}
## Configuration
This section explains the usage of options `youtube.conf` file in configuring the bot.
- `key` - Used for setting the API key. See the above section on setting up the bot.
- `number_of_results` - The maximum number of videos to show when searching
for a list of videos with the `@YouTube list <keyword>` command.
- `video_region` - The location to be used for searching.
The bot shows only the videos that are available in the given `<video_region>`
## Usage
1. `@YouTube <keyword>`
- This command search YouTube with the given keyword and gives the top result of the search.
This can also be done with the command `@YouTube top <keyword>`
- Example usage: `@YouTube funny cats` , `@YouTube top funny dogs`
![](/static/generated/bots/youtube/assets/youtube-search.png)
2. `@YouTube list <keyword>`
- This command search YouTube with the given keyword and gives a list of videos associated with the keyword.
- Example usage: `@YouTube list origami`
![](/static/generated/bots/youtube/assets/youtube-list.png)
2. If a video can't be found for a given keyword, the bot will
respond with an error message
![](/static/generated/bots/youtube/assets/youtube-not-found.png)
3. If there's a error while searching, the bot will respond with an
error message
![](/static/generated/bots/youtube/assets/youtube-error.png)

View file

@ -0,0 +1,31 @@
{
"request": {
"api_url": "https://www.googleapis.com/youtube/v3/search",
"params": {
"part": "id,snippet",
"maxResults": 1,
"key": "somethinginvalid",
"q": "test",
"alt": "json",
"type": "video",
"regionCode": "US"
}
},
"response": {
"error": {
"errors": [
{
"domain": "usageLimits",
"reason": "keyInvalid",
"message": "Bad Request"
}
],
"code": 400,
"message": "Bad Request"
}
},
"response-headers": {
"status": 400,
"content-type": "application/json; charset=utf-8"
}
}

View file

@ -0,0 +1,64 @@
{
"request": {
"api_url": "https://www.googleapis.com/youtube/v3/search",
"params": {
"part": "id,snippet",
"maxResults": 1,
"key": "12345678",
"q": "test",
"alt": "json",
"type": "video",
"regionCode": "US"
}
},
"response": {
"kind": "youtube#searchListResponse",
"etag": "\"etag\"",
"nextPageToken": "CAEQAA",
"regionCode": "IN",
"pageInfo": {
"totalResults": 1000000,
"resultsPerPage": 1
},
"items": [
{
"kind": "youtube#searchResult",
"etag": "\"etag\"",
"id": {
"kind": "youtube#video",
"videoId": "randomID"
},
"snippet": {
"publishedAt": "2016-12-24T10:30:00.000Z",
"channelId": "randomChannelID",
"title": "some random title",
"description": "This would be the description of the video",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/randomID/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/randomID/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/randomID/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "title",
"liveBroadcastContent": "none"
}
}
]
},
"response-headers": {
"status": 200,
"content-type": "application/json; charset=utf-8"
}
}

View file

@ -0,0 +1,195 @@
{
"request": {
"api_url": "https://www.googleapis.com/youtube/v3/search",
"params": {
"part": "id,snippet",
"maxResults": 5,
"key": "12345678",
"q": "marvel",
"alt": "json",
"type": "video",
"regionCode": "US"
}
},
"response": {
"kind": "youtube#searchListResponse",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/G9CmYGTc8DpRgZib1bcD0ZeBW2o\"",
"nextPageToken": "CAUQAA",
"regionCode": "US",
"pageInfo": {
"totalResults": 1000000,
"resultsPerPage": 5
},
"items": [
{
"kind": "youtube#searchResult",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/HSKDFkwbVuEgNHxkHX5mHYrnwJU\"",
"id": {
"kind": "youtube#video",
"videoId": "6ZfuNTqbHE8"
},
"snippet": {
"publishedAt": "2017-11-29T13:26:24.000Z",
"channelId": "UCvC4D8onUfXzvjTOM-dBfEA",
"title": "Marvel Studios' Avengers: Infinity War Official Trailer",
"description": "\"There was an idea…\" Avengers: Infinity War. In theaters May 4. ▻ Subscribe to Marvel: http://bit.ly/WeO3YJ Follow Marvel on Twitter: https://twitter.com/marvel Like Marvel on FaceBook:...",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/6ZfuNTqbHE8/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/6ZfuNTqbHE8/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/6ZfuNTqbHE8/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "Marvel Entertainment",
"liveBroadcastContent": "none"
}
},
{
"kind": "youtube#searchResult",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/lPJT_xBo3mCrT-XHBvtwkKLT-hw\"",
"id": {
"kind": "youtube#video",
"videoId": "xjDjIWPwcPU"
},
"snippet": {
"publishedAt": "2017-10-16T13:00:07.000Z",
"channelId": "UCvC4D8onUfXzvjTOM-dBfEA",
"title": "Marvel Studios' Black Panther - Official Trailer",
"description": "Long live the king. Watch the new trailer for Marvel Studios #BlackPanther. In theaters February 16! ▻ Subscribe to Marvel: http://bit.ly/WeO3YJ Follow Marvel on Twitter: https://twitter.com/...",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/xjDjIWPwcPU/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/xjDjIWPwcPU/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/xjDjIWPwcPU/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "Marvel Entertainment",
"liveBroadcastContent": "none"
}
},
{
"kind": "youtube#searchResult",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/vuAgL_ymZ4dpC95w3-5JcrrqERM\"",
"id": {
"kind": "youtube#video",
"videoId": "6HTPCTtkWoA"
},
"snippet": {
"publishedAt": "2017-12-07T19:00:01.000Z",
"channelId": "UCxwitsUVNzwS5XBSC5UQV8Q",
"title": "MARVEL RISING BEGINS! | The Next Generation of Marvel Heroes (EXCLUSIVE)",
"description": "With “Marvel Rising,” the next generation of Marvel heroes has arrived. Rising 2018. --- Cast: Kathleen Khavari Kamala Khan/Ms. Marvel Milana Vayntrub Doreen Green/Squirrel Girl...",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/6HTPCTtkWoA/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/6HTPCTtkWoA/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/6HTPCTtkWoA/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "Marvel HQ",
"liveBroadcastContent": "none"
}
},
{
"kind": "youtube#searchResult",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/NC5lIdIK3cIT6mlmTtw8YSUUf2A\"",
"id": {
"kind": "youtube#video",
"videoId": "-8uqxdcJ9WM"
},
"snippet": {
"publishedAt": "2017-12-07T16:00:00.000Z",
"channelId": "UCvC4D8onUfXzvjTOM-dBfEA",
"title": "Marvel Contest of Champions Taskmaster Spotlight",
"description": "Subscribe to Marvel: http://bit.ly/WeO3YJ Follow Marvel on Twitter: https://twitter.com/marvel Like Marvel on FaceBook: https://www.facebook.com/Marvel For even more news, stay...",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/-8uqxdcJ9WM/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/-8uqxdcJ9WM/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/-8uqxdcJ9WM/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "Marvel Entertainment",
"liveBroadcastContent": "none"
}
},
{
"kind": "youtube#searchResult",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/O3fss6u7H9TJ8My3sTMgB1Y4SzU\"",
"id": {
"kind": "youtube#video",
"videoId": "l7rrsGKJ_O4"
},
"snippet": {
"publishedAt": "2017-12-07T20:59:49.000Z",
"channelId": "UCesCyJp53gCYwhr8fGYEpNw",
"title": "5* Crystal Opening! SO LUCKY! - Marvel Contest Of Champions",
"description": "yoooooooooo i cannot beleive this man.... i honestly dont know why but i decided to open up a basic 5* Crystal...",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/l7rrsGKJ_O4/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/l7rrsGKJ_O4/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/l7rrsGKJ_O4/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "Lagacy69",
"liveBroadcastContent": "none"
}
}
]
},
"response-headers": {
"status": 200,
"content-type": "application/json; charset=utf-8"
}
}

View file

@ -0,0 +1,28 @@
{
"request": {
"api_url": "https://www.googleapis.com/youtube/v3/search",
"params": {
"part": "id,snippet",
"maxResults": 1,
"key": "12345678",
"q": "somethingrandomwithnoresult",
"alt": "json",
"type": "video",
"regionCode": "US"
}
},
"response": {
"kind": "youtube#searchListResponse",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/-f6JA5_OcXz2RWuH1mpAA2_9mM8\"",
"regionCode": "US",
"pageInfo": {
"totalResults": 0,
"resultsPerPage": 5
},
"items": []
},
"response-headers": {
"status": 200,
"content-type": "application/json; charset=utf-8"
}
}

View file

@ -0,0 +1,64 @@
{
"request": {
"api_url": "https://www.googleapis.com/youtube/v3/search",
"params": {
"part": "id,snippet",
"maxResults": 1,
"key": "12345678",
"q": "funny cats",
"alt": "json",
"type": "video",
"regionCode": "US"
}
},
"response": {
"kind": "youtube#searchListResponse",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/XGKpndf94WPFq2HVzaP1-nslXCQ\"",
"nextPageToken": "CAEQAA",
"regionCode": "IN",
"pageInfo": {
"totalResults": 1000000,
"resultsPerPage": 1
},
"items": [
{
"kind": "youtube#searchResult",
"etag": "\"7991kDR-QPaa9r0pePmDjBEa2h8/u2n3wez7ljBkwPSV6WkGrkhsBlI\"",
"id": {
"kind": "youtube#video",
"videoId": "5dsGWM5XGdg"
},
"snippet": {
"publishedAt": "2016-12-24T10:30:00.000Z",
"channelId": "UCKy3MG7_If9KlVuvw3rPMfw",
"title": "Cats are so funny you will die laughing - Funny cat compilation",
"description": "Cats are simply the funniest and most hilarious pets, they make us laugh all the time! Just look how all these cats & kittens play, fail, get along with dogs and ...",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/5dsGWM5XGdg/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/5dsGWM5XGdg/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/5dsGWM5XGdg/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "Tiger Productions",
"liveBroadcastContent": "none"
}
}
]
},
"response-headers": {
"status": 200,
"content-type": "application/json; charset=utf-8"
}
}

View file

@ -0,0 +1,80 @@
#!/usr/bin/env python
from __future__ import absolute_import
from unittest.mock import patch
from requests.exceptions import HTTPError, ConnectionError
from zulip_bots.test_lib import StubBotHandler, StubBotTestCase, get_bot_message_handler
from typing import Any, Union, Dict
class TestYoutubeBot(StubBotTestCase):
bot_name = "youtube"
normal_config = {'key': '12345678',
'number_of_results': '5',
'video_region': 'US'} # type: Dict[str,str]
def test_single(self) -> None:
bot_response = 'Here is what I found for `funny cats` : \n'\
'Cats are so funny you will die laughing - ' \
'Funny cat compilation - [Watch now](https://www.youtube.com/watch?v=5dsGWM5XGdg)'
with self.mock_config_info(self.normal_config), \
self.mock_http_conversation('test_single'):
self.verify_reply('funny cats', bot_response)
def test_invalid_key(self) -> None:
bot = get_bot_message_handler(self.bot_name)
bot_handler = StubBotHandler()
with self.mock_config_info({'key': 'somethinginvalid', 'number_of_results': '5', 'video_region': 'US'}), \
self.mock_http_conversation('test_invalid_key'), \
self.assertRaises(SystemExit) as se: # type: ignore
bot.initialize(bot_handler)
def test_multiple(self) -> None:
bot = get_bot_message_handler(self.bot_name)
bot_handler = StubBotHandler()
bot_response = 'Here is what I found for `marvel` : ' \
'\n * Marvel Studios\' Avengers: Infinity War Official Trailer - [Watch now](https://www.youtube.com/watch/6ZfuNTqbHE8)' \
'\n * Marvel Studios\' Black Panther - Official Trailer - [Watch now](https://www.youtube.com/watch/xjDjIWPwcPU)' \
'\n * MARVEL RISING BEGINS! | The Next Generation of Marvel Heroes (EXCLUSIVE) - [Watch now](https://www.youtube.com/watch/6HTPCTtkWoA)' \
'\n * Marvel Contest of Champions Taskmaster Spotlight - [Watch now](https://www.youtube.com/watch/-8uqxdcJ9WM)' \
'\n * 5* Crystal Opening! SO LUCKY! - Marvel Contest Of Champions - [Watch now](https://www.youtube.com/watch/l7rrsGKJ_O4)'
with self.mock_config_info(self.normal_config), \
self.mock_http_conversation('test_multiple'):
self.verify_reply('list marvel', bot_response)
def test_noresult(self) -> None:
bot_response = 'Oops ! Sorry I couldn\'t find any video for `somethingrandomwithnoresult` ' \
':slightly_frowning_face:'
with self.mock_config_info(self.normal_config), \
self.mock_http_conversation('test_noresult'):
self.verify_reply('somethingrandomwithnoresult', bot_response,)
def test_help(self) -> None:
help_content = "*Help for YouTube bot* :robot_face: : \n\n" \
"The bot responds to messages starting with @mention-bot.\n\n" \
"`@mention-bot <search terms>` will return top Youtube video for the given `<search term>`.\n" \
"`@mention-bot top <search terms>` also returns the top Youtube result.\n" \
"`@mention-bot list <search terms>` will return a list Youtube videos for the given <search term>.\n \n" \
"Example:\n" \
" * @mention-bot funny cats\n" \
" * @mention-bot list funny dogs"
with self.mock_config_info(self.normal_config), \
self.mock_http_conversation('test_keyok'):
self.verify_reply('help', help_content)
self.verify_reply('list', help_content)
self.verify_reply('help list', help_content)
self.verify_reply('top', help_content)
self.verify_reply('', help_content)
def test_connection_error(self) -> None:
with self.mock_config_info(self.normal_config), \
patch('requests.get', side_effect=ConnectionError()), \
patch('logging.exception'):
self.verify_reply('Wow !', 'Uh-Oh, couldn\'t process the request '
'right now.\nPlease again later')

View file

@ -0,0 +1,4 @@
[youtube]
key = <your key here>
number_of_results = 5
video_region = US

View file

@ -0,0 +1,133 @@
from __future__ import absolute_import
import requests
import logging
import sys
from requests.exceptions import HTTPError, ConnectionError
from typing import Dict, Any, Union, List, Tuple
commands_list = ('list', 'top', 'help')
class YoutubeHandler(object):
def usage(self) -> str:
return '''
This plugin will allow users to search
for a given search term on Youtube.
Use '@mention-bot help' to get more information on the bot usage.
'''
help_content = "*Help for YouTube bot* :robot_face: : \n\n" \
"The bot responds to messages starting with @mention-bot.\n\n" \
"`@mention-bot <search terms>` will return top Youtube video for the given `<search term>`.\n" \
"`@mention-bot top <search terms>` also returns the top Youtube result.\n" \
"`@mention-bot list <search terms>` will return a list Youtube videos for the given <search term>.\n \n" \
"Example:\n" \
" * @mention-bot funny cats\n" \
" * @mention-bot list funny dogs"
def initialize(self, bot_handler: Any) -> None:
self.config_info = bot_handler.get_config_info('youtube')
# Check if API key is valid. If it is not valid, don't run the bot.
try:
search_youtube('test', self.config_info['key'], self.config_info['video_region'])
except HTTPError as e:
if (e.response.json()['error']['errors'][0]['reason'] == 'keyInvalid'):
logging.error('Invalid key.'
'Follow the instructions in doc.md for setting API key.')
sys.exit(1)
else:
raise
except ConnectionError:
logging.warning('Bad connection')
def handle_message(self, message: Dict[str, str], bot_handler: Any) -> None:
if message['content'] == '' or message['content'] == 'help':
bot_handler.send_reply(message, self.help_content)
else:
cmd, query = get_command_query(message)
bot_response = get_bot_response(query,
cmd,
self.config_info)
logging.info(bot_response.format())
bot_handler.send_reply(message, bot_response)
def search_youtube(query: str, key: str,
region: str, max_results: int = 1) -> List[List[str]]:
videos = []
params = {
'part': 'id,snippet',
'maxResults': max_results,
'key': key,
'q': query,
'alt': 'json',
'type': 'video',
'regionCode': region} # type: Dict[str, Union[str, int]]
url = 'https://www.googleapis.com/youtube/v3/search'
try:
r = requests.get(url, params=params)
except ConnectionError as e: # Usually triggered by bad connection.
logging.exception('Bad connection')
raise
r.raise_for_status()
search_response = r.json()
# Add each result to the appropriate list, and then display the lists of
# matching videos, channels, and playlists.
for search_result in search_response.get('items', []):
if search_result['id']['kind'] == 'youtube#video':
videos.append([search_result['snippet']['title'],
search_result['id']['videoId']])
return videos
def get_command_query(message: Dict[str, str]) -> Tuple[Union[None, str], str]:
blocks = message['content'].lower().split()
command = blocks[0]
if command in commands_list:
query = message['content'][len(command) + 1:].lstrip()
return command, query
else:
return None, message['content']
def get_bot_response(query: Union[str, None], command: Union[str, None], config_info: Dict[str, str]) -> str:
key = config_info['key']
max_results = int(config_info['number_of_results'])
region = config_info['video_region']
reply = 'Here is what I found for `' + query + '` : '
video_list = [] # type: List[List[str]]
try:
if query == '' or query is None:
return YoutubeHandler.help_content
if command is None or command == 'top':
video_list = search_youtube(query, key, region)
elif command == 'list':
video_list = search_youtube(query, key, region, max_results)
elif command == 'help':
return YoutubeHandler.help_content
except (ConnectionError, HTTPError):
return 'Uh-Oh, couldn\'t process the request ' \
'right now.\nPlease again later'
if len(video_list) == 0:
return 'Oops ! Sorry I couldn\'t find any video for `' + query + '` :slightly_frowning_face:'
elif len(video_list) == 1:
return (reply + '\n%s - [Watch now](https://www.youtube.com/watch?v=%s)' % (video_list[0][0], video_list[0][1])).strip()
for title, id in video_list:
reply = reply + \
'\n * %s - [Watch now](https://www.youtube.com/watch/%s)' % (title, id)
# Using link https://www.youtube.com/watch/<id> to
# prevent showing multiple previews
return reply
handler_class = YoutubeHandler

Binary file not shown.

Before

Width:  |  Height:  |  Size: 81 KiB

View file

@ -1,11 +0,0 @@
# Youtube bot
Youtube bot is a Zulip bot that can fetch first video from youtube
search results for a specified term. To use youtube bot you can simply
call it with `@mention-bot` followed by a command. Like this:
```
@mention-bot <search term>
```
![example usage](assets/screen.png)

View file

@ -1,31 +0,0 @@
# See readme.md for instructions on running this bot.
import requests
from bs4 import BeautifulSoup
class YoutubeHandler(object):
def usage(self):
return '''
This bot will return the first Youtube search result for the give query.
'''
def handle_message(self, message, bot_handler):
help_content = '''
To use the, Youtube Bot send `@mention-bot search terms`
Example:
@mention-bot funny cats
'''.strip()
if message['content'] == '':
bot_handler.send_reply(message, help_content)
else:
text_to_search = message['content']
url = "https://www.youtube.com/results?search_query=" + text_to_search
r = requests.get(url)
soup = BeautifulSoup(r.text, 'lxml')
video_id = soup.find(attrs={'class': 'yt-uix-tile-link'})
try:
link = 'https://www.youtube.com' + video_id['href']
bot_handler.send_reply(message, link)
except TypeError:
bot_handler.send_reply(message, 'No video found for specified search terms')
handler_class = YoutubeHandler