xenofem/python-zulip-api
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity

api: Move the API package to a dedicated subdirectory.

Browse source 
In order to keep all three packages (zulip, zulip_bots,
zulip_botserver) in the same repo, all package files must now
be nested one level deeper.

For instance, python-zulip-api/zulip_bots/zulip_bots/bots/, instead
of python-zulip-api/zulip_bots/bots/.
This commit is contained in:
Eeshan Garg 2017-07-18 01:49:51 -02:30
parent 879f44ab3a
commit 3d0f7955b6
59 changed files with 186 additions and 192 deletions
Download patch file Download diff file Expand all files Collapse all files

11
zulip/MANIFEST.in Normal file
View file

@ -0,0 +1,11 @@
recursive-include integrations *
include README.md
include examples/zuliprc
include examples/send-message
include examples/subscribe
include examples/get-public-streams
include examples/unsubscribe
include examples/list-members
include examples/list-subscriptions
include examples/print-messages
include examples/recent-messages

177
zulip/README.md Normal file
View file

@ -0,0 +1,177 @@
#### Dependencies
The [Zulip API](https://zulipchat.com/api) Python bindings require the
following Python libraries:
* requests (version >= 0.12.1)
* simplejson
* six
* typing (version >= 3.5.2.2)
#### Installing
This package uses distutils, so you can just run:
python setup.py install
#### Using the API
For now, the only fully supported API operation is sending a message.
The other API queries work, but are under active development, so
please make sure we know you're using them so that we can notify you
as we make any changes to them.
The easiest way to use these API bindings is to base your tools off
of the example tools under examples/ in this distribution.
If you place your API key in the config file `~/.zuliprc` the Python
API bindings will automatically read it in. The format of the config
file is as follows:
[api]
key=<api key from the web interface>
email=<your email address>
site=<your Zulip server's URI>
insecure=<true or false, true means do not verify the server certificate>
cert_bundle=<path to a file containing CA or server certificates to trust>
If omitted, these settings have the following defaults:
insecure=false
cert_bundle=<the default CA bundle trusted by Python>
Alternatively, you may explicitly use "--user", "--api-key", and
`--site` in our examples, which is especially useful when testing. If
you are running several bots which share a home directory, we
recommend using `--config` to specify the path to the `zuliprc` file
for a specific bot. Finally, you can control the defaults for all of
these variables using the environment variables `ZULIP_CONFIG`,
`ZULIP_API_KEY`, `ZULIP_EMAIL`, `ZULIP_SITE`, `ZULIP_CERT`,
`ZULIP_CERT_KEY`, and `ZULIP_CERT_BUNDLE`. Command-line options take
precedence over environment variables take precedence over the config
files.
The command line equivalents for other configuration options are:
--insecure
--cert-bundle=<file>
You can obtain your Zulip API key, create bots, and manage bots all
from your Zulip settings page; with current Zulip there's also a
button to download a `zuliprc` file for your account/server pair.
A typical simple bot sending API messages will look as follows:
At the top of the file:
# Make sure the Zulip API distribution's root directory is in sys.path, then:
import zulip
zulip_client = zulip.Client(email="your-bot@example.com", client="MyTestClient/0.1")
When you want to send a message:
message = {
"type": "stream",
"to": ["support"],
"subject": "your subject",
"content": "your content",
}
zulip_client.send_message(message)
If you are parsing arguments, you may find it useful to use Zulip's
option group; see any of our API examples for details on how to do this.
Additional examples:
client.send_message({'type': 'stream', 'content': 'Zulip rules!',
'subject': 'feedback', 'to': ['support']})
client.send_message({'type': 'private', 'content': 'Zulip rules!',
'to': ['user1@example.com', 'user2@example.com']})
send_message() returns a dict guaranteed to contain the following
keys: msg, result. For successful calls, result will be "success" and
msg will be the empty string. On error, result will be "error" and
msg will describe what went wrong.
#### Examples
The API bindings package comes with several nice example scripts that
show how to use the APIs; they are installed as part of the API
bindings bundle.
#### Logging
The Zulip API comes with a ZulipStream class which can be used with the
logging module:
```
import zulip
import logging
stream = zulip.ZulipStream(type="stream", to=["support"], subject="your subject")
logger = logging.getLogger("your_logger")
logger.addHandler(logging.StreamHandler(stream))
logger.setLevel(logging.DEBUG)
logger.info("This is an INFO test.")
logger.debug("This is a DEBUG test.")
logger.warn("This is a WARN test.")
logger.error("This is a ERROR test.")
```
#### Sending messages
You can use the included `zulip-send` script to send messages via the
API directly from existing scripts.
zulip-send hamlet@example.com cordelia@example.com -m \
"Conscience doth make cowards of us all."
Alternatively, if you don't want to use your ~/.zuliprc file:
zulip-send --user shakespeare-bot@example.com \
--api-key a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 \
--site https://zulip.example.com \
hamlet@example.com cordelia@example.com -m \
"Conscience doth make cowards of us all."
#### Working with an untrusted server certificate
If your server has either a self-signed certificate, or a certificate signed
by a CA that you don't wish to globally trust then by default the API will
fail with an SSL verification error.
You can add `insecure=true` to your .zuliprc file.
[api]
site=https://zulip.example.com
insecure=true
This disables verification of the server certificate, so connections are
encrypted but unauthenticated. This is not secure, but may be good enough
for a development environment.
You can explicitly trust the server certificate using `cert_bundle=<filename>`
in your .zuliprc file.
[api]
site=https://zulip.example.com
cert_bundle=/home/bots/certs/zulip.example.com.crt
You can also explicitly trust a different set of Certificate Authorities from
the default bundle that is trusted by Python. For example to trust a company
internal CA.
[api]
site=https://zulip.example.com
cert_bundle=/home/bots/certs/example.com.ca-bundle
Save the server certificate (or the CA certificate) in its own file,
converting to PEM format first if necessary.
Verify that the certificate you have saved is the same as the one on the
server.
The `cert_bundle` option trusts the server / CA certificate only for
interaction with the zulip site, and is relatively secure.
Note that a certificate bundle is merely one or more certificates combined
into a single file.

55
zulip/examples/create-user Executable file
View file

@ -0,0 +1,55 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012-2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
from os import path
import optparse
usage = """create-user --new-email=<email address> --new-password=<password> --new-full-name=<full name> --new-short-name=<short name> [options]
Create a user. You must be a realm admin to use this API, and the user
will be created in your realm.
Example: create-user --site=http://localhost:9991 --user=rwbarton@example.com --new-email=jarthur@example.com --new-password=random17 --new-full-name 'J. Arthur Random' --new-short-name='jarthur'
"""
sys.path.append(path.join(path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
parser.add_option('--new-email')
parser.add_option('--new-password')
parser.add_option('--new-full-name')
parser.add_option('--new-short-name')
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
print(client.create_user({
'email': options.new_email,
'password': options.new_password,
'full_name': options.new_full_name,
'short_name': options.new_short_name
}))

57
zulip/examples/edit-message Executable file
View file

@ -0,0 +1,57 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
usage = """edit-message [options] --message=<msg_id> --subject=<new subject> --content=<new content> --user=<sender's email address> --api-key=<sender's api key>
Edits a message that you sent
Example: edit-message --message-id="348135" --subject="my subject" --content="test message" --user=othello-bot@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option('--message-id', default="")
parser.add_option('--subject', default="")
parser.add_option('--content', default="")
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
message_data = {
"message_id": options.message_id,
}
if options.subject != "":
message_data["subject"] = options.subject
if options.content != "":
message_data["content"] = options.content
print(client.update_message(message_data))

44
zulip/examples/get-presence Executable file
View file

@ -0,0 +1,44 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012-2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
from os import path
import optparse
usage = """get-presence --email=<email address> [options]
Get presence data for another user.
"""
sys.path.append(path.join(path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
parser.add_option('--email')
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
print(client.get_presence(options.email))

47
zulip/examples/get-public-streams Executable file
View file

@ -0,0 +1,47 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
usage = """get-public-streams --user=<bot's email address> --api-key=<bot's api key> [options]
Prints out all the public streams in the realm.
Example: get-public-streams --user=othello-bot@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
print(client.get_streams(include_public=True, include_subscribed=False))

46
zulip/examples/list-members Executable file
View file

@ -0,0 +1,46 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
usage = """list-members --user=<bot's email address> --api-key=<bot's api key> [options]
List the names and e-mail addresses of the people in your realm.
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
for user in client.get_members()["members"]:
print(user["full_name"], user["email"])

46
zulip/examples/list-subscriptions Executable file
View file

@ -0,0 +1,46 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
usage = """list-subscriptions --user=<bot's email address> --api-key=<bot's api key> [options]
Prints out a list of the user's subscriptions.
Example: list-subscriptions --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
print(client.list_subscriptions())

55
zulip/examples/print-events Executable file
View file

@ -0,0 +1,55 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
from typing import Any, Dict
usage = """print-events --user=<bot's email address> --api-key=<bot's api key> [options]
Prints out certain events received by the indicated bot or user matching the filter below.
Example: print-events --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
def print_event(event):
# type: (Dict[str, Any]) -> None
print(event)
# This is a blocking call, and will continuously poll for new events
# Note also the filter here is messages to the stream Denmark; if you
# don't specify event_types it'll print all events.
client.call_on_each_event(print_event, event_types=["message"], narrow=[["stream", "Denmark"]])

53
zulip/examples/print-messages Executable file
View file

@ -0,0 +1,53 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
from typing import Any, Dict
usage = """print-messages --user=<bot's email address> --api-key=<bot's api key> [options]
Prints out each message received by the indicated bot or user.
Example: print-messages --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
def print_message(message):
# type: (Dict[str, Any]) -> None
print(message)
# This is a blocking call, and will continuously poll for new messages
client.call_on_each_message(print_message)

66
zulip/examples/recent-messages Executable file
View file

@ -0,0 +1,66 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import json
import optparse
usage = """recent-messages [options] --count=<no. of previous messages> --user=<sender's email address> --api-key=<sender's api key>
Prints out last count messages received by the indicated bot or user
Example: recent-messages --count=101 --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option('--count', default=100)
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
request = {
'narrow': [["stream", "Denmark"]],
'num_before': options.count,
'num_after': 0,
'anchor': 1000000000,
'apply_markdown': False
}
old_messages = client.call_endpoint(
url='messages',
method='GET',
request=request,
)
if 'messages' in old_messages:
for message in old_messages['messages']:
print(json.dumps(message, indent=4))
else:
print([])

58
zulip/examples/send-message Executable file
View file

@ -0,0 +1,58 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
usage = """send-message --user=<bot's email address> --api-key=<bot's api key> [options] <recipients>
Sends a test message to the specified recipients.
Example: send-message --user=your-bot@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 --type=stream commits --subject="my subject" --message="test message"
Example: send-message --user=your-bot@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 user1@example.com user2@example.com
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
parser = optparse.OptionParser(usage=usage)
parser.add_option('--subject', default="test")
parser.add_option('--message', default="test message")
parser.add_option('--type', default='private')
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
if len(args) == 0:
parser.error("You must specify recipients")
client = zulip.init_from_options(options)
message_data = {
"type": options.type,
"content": options.message,
"subject": options.subject,
"to": args,
}
print(client.send_message(message_data))

53
zulip/examples/subscribe Executable file
View file

@ -0,0 +1,53 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
usage = """subscribe --user=<bot's email address> --api-key=<bot's api key> [options] --streams=<streams>
Ensures the user is subscribed to the listed streams.
Examples: subscribe --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 --streams=foo
subscribe --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 --streams='foo bar'
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
parser.add_option('--streams', default='')
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
if options.streams == "":
print("Usage:", parser.usage, file=sys.stderr)
sys.exit(1)
print(client.add_subscriptions([{"name": stream_name} for stream_name in
options.streams.split()]))

52
zulip/examples/unsubscribe Executable file
View file

@ -0,0 +1,52 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import sys
import os
import optparse
usage = """unsubscribe --user=<bot's email address> --api-key=<bot's api key> [options] --streams=<streams>
Ensures the user is not subscribed to the listed streams.
Examples: unsubscribe --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 --streams=foo
unsubscribe --user=username@example.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 --streams='foo bar'
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
"""
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
parser = optparse.OptionParser(usage=usage)
parser.add_option_group(zulip.generate_option_group(parser))
parser.add_option('--streams', default='')
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
if options.streams == "":
print("Usage:", parser.usage, file=sys.stderr)
sys.exit(1)
print(client.remove_subscriptions(options.streams.split()))

66
zulip/examples/upload-file Executable file
View file

@ -0,0 +1,66 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Copyright © 2012-2017 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import importlib
import optparse
import sys
from six.moves import StringIO as _StringIO
sys.path.insert(0, './api')
from typing import IO
import zulip
class StringIO(_StringIO):
name = '' # https://github.com/python/typeshed/issues/598
usage = """upload-file --user=<user's email address> --api-key=<user's api key> [options]
Upload a file, and print the corresponding URI.
Example: upload-file --user=cordelia@zulip.com --api-key=a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 --file-path=cat.png
You can omit --user and --api-key arguments if you have a properly set up ~/.zuliprc
If no --file-path is specified, a placeholder text file will be used instead.
"""
parser = optparse.OptionParser(usage=usage)
parser.add_option('--file-path')
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
client = zulip.init_from_options(options)
file = None # type: IO
if options.file_path:
file = open(options.file_path, 'rb')
else:
file = StringIO('This is a test file.')
file.name = 'test.txt'
response = client.upload_file(file)
try:
print('File URI: {}'.format(response['uri']))
except KeyError:
print('Error! API response was: {}'.format(response))

4
zulip/examples/zuliprc Normal file
View file

@ -0,0 +1,4 @@
; Save this file as ~/.zuliprc
[api]
key=<your bot's api key from the web interface>
email=<your bot's email address>

60
zulip/integrations/codebase/zulip_codebase_config.py Normal file
View file

@ -0,0 +1,60 @@
# -*- coding: utf-8 -*-
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# Change these values to configure authentication for your codebase account
# Note that this is the Codebase API Username, found in the Settings page
# for your account
CODEBASE_API_USERNAME = "foo@example.com"
CODEBASE_API_KEY = "1234561234567abcdef"
# The URL of your codebase setup
CODEBASE_ROOT_URL = "https://YOUR_COMPANY.codebasehq.com"
# When initially started, how many hours of messages to include.
# Note that the Codebase API only returns the 20 latest events,
# if you have more than 20 events that fit within this window,
# earlier ones may be lost
CODEBASE_INITIAL_HISTORY_HOURS = 12
# Change these values to configure Zulip authentication for the plugin
ZULIP_USER = "codebase-bot@example.com"
ZULIP_API_KEY = "0123456789abcdef0123456789abcdef"
# The streams to send commit information and ticket information to
ZULIP_COMMITS_STREAM_NAME = "codebase"
ZULIP_TICKETS_STREAM_NAME = "tickets"
# If properly installed, the Zulip API should be in your import
# path, but if not, set a custom path below
ZULIP_API_PATH = None
# Set this to your Zulip API server URI
ZULIP_SITE = "https://zulip.example.com"
# If you wish to log to a file rather than stdout/stderr,
# please fill this out your desired path
LOG_FILE = None
# This file is used to resume this mirror in case the script shuts down.
# It is required and needs to be writeable.
RESUME_FILE = "/var/tmp/zulip_codebase.state"

333
zulip/integrations/codebase/zulip_codebase_mirror Executable file
View file

@ -0,0 +1,333 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Zulip mirror of Codebase HQ activity
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#
# The "zulip_codebase_mirror" script is run continuously, possibly on a work
# computer or preferably on a server.
#
# When restarted, it will attempt to pick up where it left off.
#
# python-dateutil is a dependency for this script.
from __future__ import print_function
from __future__ import absolute_import
import requests
import logging
import pytz
import time
import sys
import os
from datetime import datetime, timedelta
try:
import dateutil.parser
except ImportError as e:
print(e, file=sys.stderr)
print("Please install the python-dateutil package.", file=sys.stderr)
exit(1)
sys.path.insert(0, os.path.dirname(__file__))
import zulip_codebase_config as config
VERSION = "0.9"
if config.ZULIP_API_PATH is not None:
sys.path.append(config.ZULIP_API_PATH)
import six
import zulip
from typing import Any, List, Dict, Optional
client = zulip.Client(
email=config.ZULIP_USER,
site=config.ZULIP_SITE,
api_key=config.ZULIP_API_KEY,
client="ZulipCodebase/" + VERSION)
user_agent = "Codebase To Zulip Mirroring script (zulip-devel@googlegroups.com)"
# find some form of JSON loader/dumper, with a preference order for speed.
json_implementations = ['ujson', 'cjson', 'simplejson', 'json']
while len(json_implementations):
try:
json = __import__(json_implementations.pop(0))
break
except ImportError:
continue
def make_api_call(path):
# type: (str) -> Optional[List[Dict[str, Any]]]
response = requests.get("https://api3.codebasehq.com/%s" % (path,),
auth=(config.CODEBASE_API_USERNAME, config.CODEBASE_API_KEY),
params={'raw': 'True'},
headers = {"User-Agent": user_agent,
"Content-Type": "application/json",
"Accept": "application/json"})
if response.status_code == 200:
return json.loads(response.text)
if response.status_code >= 500:
logging.error(str(response.status_code))
return None
if response.status_code == 403:
logging.error("Bad authorization from Codebase. Please check your credentials")
sys.exit(-1)
else:
logging.warn("Found non-success response status code: %s %s" % (response.status_code, response.text))
return None
def make_url(path):
# type: (str) -> str
return "%s/%s" % (config.CODEBASE_ROOT_URL, path)
def handle_event(event):
# type: (Dict[str, Any]) -> None
event = event['event']
event_type = event['type']
actor_name = event['actor_name']
raw_props = event.get('raw_properties', {})
project_link = raw_props.get('project_permalink')
subject = None
content = None
if event_type == 'repository_creation':
stream = config.ZULIP_COMMITS_STREAM_NAME
project_name = raw_props.get('name')
project_repo_type = raw_props.get('scm_type')
url = make_url("projects/%s" % (project_link,))
scm = "of type %s" % (project_repo_type,) if project_repo_type else ""
subject = "Repository %s Created" % (project_name,)
content = "%s created a new repository %s [%s](%s)" % (actor_name, scm, project_name, url)
elif event_type == 'push':
stream = config.ZULIP_COMMITS_STREAM_NAME
num_commits = raw_props.get('commits_count')
branch = raw_props.get('ref_name')
project = raw_props.get('project_name')
repo_link = raw_props.get('repository_permalink')
deleted_ref = raw_props.get('deleted_ref')
new_ref = raw_props.get('new_ref')
subject = "Push to %s on %s" % (branch, project)
if deleted_ref:
content = "%s deleted branch %s from %s" % (actor_name, branch, project)
else:
if new_ref:
branch = "new branch %s" % (branch,)
content = ("%s pushed %s commit(s) to %s in project %s:\n\n" %
(actor_name, num_commits, branch, project))
for commit in raw_props.get('commits'):
ref = commit.get('ref')
url = make_url("projects/%s/repositories/%s/commit/%s" % (project_link, repo_link, ref))
message = commit.get('message')
content += "* [%s](%s): %s\n" % (ref, url, message)
elif event_type == 'ticketing_ticket':
stream = config.ZULIP_TICKETS_STREAM_NAME
num = raw_props.get('number')
name = raw_props.get('subject')
assignee = raw_props.get('assignee')
priority = raw_props.get('priority')
url = make_url("projects/%s/tickets/%s" % (project_link, num))
if assignee is None:
assignee = "no one"
subject = "#%s: %s" % (num, name)
content = ("""%s created a new ticket [#%s](%s) priority **%s** assigned to %s:\n\n~~~ quote\n %s""" %
(actor_name, num, url, priority, assignee, name))
elif event_type == 'ticketing_note':
stream = config.ZULIP_TICKETS_STREAM_NAME
num = raw_props.get('number')
name = raw_props.get('subject')
body = raw_props.get('content')
changes = raw_props.get('changes')
url = make_url("projects/%s/tickets/%s" % (project_link, num))
subject = "#%s: %s" % (num, name)
content = ""
if body is not None and len(body) > 0:
content = "%s added a comment to ticket [#%s](%s):\n\n~~~ quote\n%s\n\n" % (actor_name, num, url, body)
if 'status_id' in changes:
status_change = changes.get('status_id')
content += "Status changed from **%s** to **%s**\n\n" % (status_change[0], status_change[1])
elif event_type == 'ticketing_milestone':
stream = config.ZULIP_TICKETS_STREAM_NAME
name = raw_props.get('name')
identifier = raw_props.get('identifier')
url = make_url("projects/%s/milestone/%s" % (project_link, identifier))
subject = name
content = "%s created a new milestone [%s](%s)" % (actor_name, name, url)
elif event_type == 'comment':
stream = config.ZULIP_COMMITS_STREAM_NAME
comment = raw_props.get('content')
commit = raw_props.get('commit_ref')
# If there's a commit id, it's a comment to a commit
if commit:
repo_link = raw_props.get('repository_permalink')
url = make_url('projects/%s/repositories/%s/commit/%s' % (project_link, repo_link, commit))
subject = "%s commented on %s" % (actor_name, commit)
content = "%s commented on [%s](%s):\n\n~~~ quote\n%s" % (actor_name, commit, url, comment)
else:
# Otherwise, this is a Discussion item, and handle it
subj = raw_props.get("subject")
category = raw_props.get("category")
comment_content = raw_props.get("content")
subject = "Discussion: %s" % (subj,)
if category:
format_str = "%s started a new discussion in %s:\n\n~~~ quote\n%s\n~~~"
content = format_str % (actor_name, category, comment_content)
else:
content = "%s posted:\n\n~~~ quote\n%s\n~~~" % (actor_name, comment_content)
elif event_type == 'deployment':
stream = config.ZULIP_COMMITS_STREAM_NAME
start_ref = raw_props.get('start_ref')
end_ref = raw_props.get('end_ref')
environment = raw_props.get('environment')
servers = raw_props.get('servers')
repo_link = raw_props.get('repository_permalink')
start_ref_url = make_url("projects/%s/repositories/%s/commit/%s" % (project_link, repo_link, start_ref))
end_ref_url = make_url("projects/%s/repositories/%s/commit/%s" % (project_link, repo_link, end_ref))
between_url = make_url("projects/%s/repositories/%s/compare/%s...%s" % (
project_link, repo_link, start_ref, end_ref))
subject = "Deployment to %s" % (environment,)
content = ("%s deployed [%s](%s) [through](%s) [%s](%s) to the **%s** environment." %
(actor_name, start_ref, start_ref_url, between_url, end_ref, end_ref_url, environment))
if servers is not None:
content += "\n\nServers deployed to: %s" % (", ".join(["`%s`" % (server,) for server in servers]))
elif event_type == 'named_tree':
# Docs say named_tree type used for new/deleting branches and tags,
# but experimental testing showed that they were all sent as 'push' events
pass
elif event_type == 'wiki_page':
logging.warn("Wiki page notifications not yet implemented")
elif event_type == 'sprint_creation':
logging.warn("Sprint notifications not yet implemented")
elif event_type == 'sprint_ended':
logging.warn("Sprint notifications not yet implemented")
else:
logging.info("Unknown event type %s, ignoring!" % (event_type,))
if subject and content:
if len(subject) > 60:
subject = subject[:57].rstrip() + '...'
res = client.send_message({"type": "stream",
"to": stream,
"subject": subject,
"content": content})
if res['result'] == 'success':
logging.info("Successfully sent Zulip with id: %s" % (res['id']))
else:
logging.warn("Failed to send Zulip: %s %s" % (res['result'], res['msg']))
# the main run loop for this mirror script
def run_mirror():
# type: () -> None
# we should have the right (write) permissions on the resume file, as seen
# in check_permissions, but it may still be empty or corrupted
def default_since():
# type: () -> datetime
return datetime.now(tz=pytz.utc) - timedelta(hours=config.CODEBASE_INITIAL_HISTORY_HOURS)
try:
with open(config.RESUME_FILE) as f:
timestamp = f.read()
if timestamp == '':
since = default_since()
else:
since = datetime.fromtimestamp(float(timestamp), tz=pytz.utc)
except (ValueError, IOError) as e:
logging.warn("Could not open resume file: %s" % (str(e)))
since = default_since()
try:
sleepInterval = 1
while True:
events = make_api_call("activity")[::-1]
if events is not None:
sleepInterval = 1
for event in events:
timestamp = event.get('event', {}).get('timestamp', '')
event_date = dateutil.parser.parse(timestamp)
if event_date > since:
handle_event(event)
since = event_date
else:
# back off a bit
if sleepInterval < 22:
sleepInterval += 4
time.sleep(sleepInterval)
except KeyboardInterrupt:
open(config.RESUME_FILE, 'w').write(since.strftime("%s"))
logging.info("Shutting down Codebase mirror")
# void function that checks the permissions of the files this script needs.
def check_permissions():
# type: () -> None
# check that the log file can be written
if config.LOG_FILE:
try:
open(config.LOG_FILE, "w")
except IOError as e:
sys.stderr.write("Could not open up log for writing:")
sys.stderr.write(str(e))
# check that the resume file can be written (this creates if it doesn't exist)
try:
open(config.RESUME_FILE, "a+")
except IOError as e:
sys.stderr.write("Could not open up the file %s for reading and writing" % (config.RESUME_FILE,))
sys.stderr.write(str(e))
if __name__ == "__main__":
if not isinstance(config.RESUME_FILE, six.string_types):
sys.stderr.write("RESUME_FILE path not given; refusing to continue")
check_permissions()
if config.LOG_FILE:
logging.basicConfig(filename=config.LOG_FILE, level=logging.WARNING)
else:
logging.basicConfig(level=logging.WARNING)
run_mirror()

120
zulip/integrations/git/post-receive Executable file
View file

@ -0,0 +1,120 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Zulip notification post-receive hook.
# Copyright © 2012-2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#
# The "post-receive" script is run after receive-pack has accepted a pack
# and the repository has been updated. It is passed arguments in through
# stdin in the form
# <oldrev> <newrev> <refname>
# For example:
# aa453216d1b3e49e7f6f98441fa56946ddcd6a20 68f7abf4e6f922807889f52bc043ecd31b79f814 refs/heads/master
from __future__ import absolute_import
from typing import Text
import os
import sys
import subprocess
import os.path
sys.path.insert(0, os.path.dirname(__file__))
import zulip_git_config as config
VERSION = "0.9"
if config.ZULIP_API_PATH is not None:
sys.path.append(config.ZULIP_API_PATH)
import zulip
client = zulip.Client(
email=config.ZULIP_USER,
site=config.ZULIP_SITE,
api_key=config.ZULIP_API_KEY,
client="ZulipGit/" + VERSION)
def git_repository_name():
# type: () -> Text
output = subprocess.check_output(["git", "rev-parse", "--is-bare-repository"])
if output.strip() == "true":
return os.path.basename(os.getcwd())[:-len(".git")]
else:
return os.path.basename(os.path.dirname(os.getcwd()))
def git_commit_range(oldrev, newrev):
# type: (str, str) -> str
log_cmd = ["git", "log", "--reverse",
"--pretty=%aE %H %s", "%s..%s" % (oldrev, newrev)]
commits = ''
for ln in subprocess.check_output(log_cmd).splitlines():
author_email, commit_id, subject = ln.split(None, 2)
if hasattr(config, "format_commit_message"):
commits += config.format_commit_message(author_email, subject, commit_id)
else:
commits += '!avatar(%s) %s\n' % (author_email, subject)
return commits
def send_bot_message(oldrev, newrev, refname):
# type: (str, str, str) -> None
repo_name = git_repository_name()
branch = refname.replace('refs/heads/', '')
destination = config.commit_notice_destination(repo_name, branch, newrev)
if destination is None:
# Don't forward the notice anywhere
return
new_head = newrev[:12]
old_head = oldrev[:12]
if (oldrev == '0000000000000000000000000000000000000000' or
newrev == '0000000000000000000000000000000000000000'):
# New branch pushed or old branch removed
added = ''
removed = ''
else:
added = git_commit_range(oldrev, newrev)
removed = git_commit_range(newrev, oldrev)
if oldrev == '0000000000000000000000000000000000000000':
message = '`%s` was pushed to new branch `%s`' % (new_head, branch)
elif newrev == '0000000000000000000000000000000000000000':
message = 'branch `%s` was removed (was `%s`)' % (branch, old_head)
elif removed:
message = '`%s` was pushed to `%s`, **REMOVING**:\n\n%s' % (new_head, branch, removed)
if added:
message += '\n**and adding**:\n\n' + added
message += '\n**A HISTORY REWRITE HAS OCCURRED!**'
message += '\n@everyone: Please check your local branches to deal with this.'
elif added:
message = '`%s` was deployed to `%s` with:\n\n%s' % (new_head, branch, added)
else:
message = '`%s` was pushed to `%s`... but nothing changed?' % (new_head, branch)
message_data = {
"type": "stream",
"to": destination["stream"],
"subject": destination["subject"],
"content": message,
}
client.send_message(message_data)
for ln in sys.stdin:
oldrev, newrev, refname = ln.strip().split()
send_bot_message(oldrev, newrev, refname)

64
zulip/integrations/git/zulip_git_config.py Normal file
View file

@ -0,0 +1,64 @@
# -*- coding: utf-8 -*-
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# Change these values to configure authentication for the plugin
ZULIP_USER = "git-bot@example.com"
ZULIP_API_KEY = "0123456789abcdef0123456789abcdef"
# commit_notice_destination() lets you customize where commit notices
# are sent to with the full power of a Python function.
#
# It takes the following arguments:
# * repo = the name of the git repository
# * branch = the name of the branch that was pushed to
# * commit = the commit id
#
# Returns a dictionary encoding the stream and subject to send the
# notification to (or None to send no notification).
#
# The default code below will send every commit pushed to "master" to
# * stream "commits"
# * topic "master"
# And similarly for branch "test-post-receive" (for use when testing).
def commit_notice_destination(repo, branch, commit):
if branch in ["master", "test-post-receive"]:
return dict(stream = "commits",
subject = u"%s" % (branch,))
# Return None for cases where you don't want a notice sent
return None
# Modify this function to change how commits are displayed; the most
# common customization is to include a link to the commit in your
# graphical repository viewer, e.g.
#
# return '!avatar(%s) [%s](https://example.com/commits/%s)\n' % (author, subject, commit_id)
def format_commit_message(author, subject, commit_id):
return '!avatar(%s) %s\n' % (author, subject)
## If properly installed, the Zulip API should be in your import
## path, but if not, set a custom path below
ZULIP_API_PATH = None
# Set this to your Zulip server's API URI
ZULIP_SITE = "https://zulip.example.com"

56
zulip/integrations/google/get-google-credentials Normal file
View file

@ -0,0 +1,56 @@
#!/usr/bin/env python
from __future__ import print_function
import datetime
import httplib2
import os
from oauth2client import client
from oauth2client import tools
from oauth2client.file import Storage
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
# If modifying these scopes, delete your previously saved credentials
# at zulip/bots/gcal/
# NOTE: When adding more scopes, add them after the previous one in the same field, with a space
# seperating them.
SCOPES = 'https://www.googleapis.com/auth/calendar.readonly'
# This file contains the information that google uses to figure out which application is requesting
# this client's data.
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Zulip Calendar Bot'
HOME_DIR = os.path.expanduser('~')
def get_credentials():
# type: () -> client.Credentials
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
credential_path = os.path.join(HOME_DIR,
'google-credentials.json')
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(os.path.join(HOME_DIR, CLIENT_SECRET_FILE), SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
# This attempts to open an authorization page in the default web browser, and asks the user
# to grant the bot access to their data. If the user grants permission, the run_flow()
# function returns new credentials.
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
get_credentials()

200
zulip/integrations/google/google-calendar Executable file
View file

@ -0,0 +1,200 @@
#!/usr/bin/env python
#
# This script depends on python-dateutil and python-pytz for properly handling
# times and time zones of calendar events.
from __future__ import print_function
import datetime
import dateutil.parser
import httplib2
import itertools
import logging
import optparse
import os
import pytz
from six.moves import urllib
import sys
import time
import traceback
from typing import List, Set, Tuple, Iterable, Optional
from oauth2client import client, tools
from oauth2client.file import Storage
try:
from googleapiclient import discovery
except ImportError:
logging.exception('Install google-api-python-client')
sys.path.append(os.path.join(os.path.dirname(__file__), '../../'))
import zulip
SCOPES = 'https://www.googleapis.com/auth/calendar.readonly'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Zulip'
HOME_DIR = os.path.expanduser('~')
# Our cached view of the calendar, updated periodically.
events = [] # type: List[Tuple[int, datetime.datetime, str]]
# Unique keys for events we've already sent, so we don't remind twice.
sent = set() # type: Set[Tuple[int, datetime.datetime]]
sys.path.append(os.path.dirname(__file__))
parser = optparse.OptionParser(r"""
%prog \
--user foo@zulip.com \
--calendar calendarID@example.calendar.google.com
This integration can be used to send yourself reminders, on Zulip, of Google Calendar Events.
Before running this integration make sure you run the get-google-credentials file to give Zulip
access to certain aspects of your Google Account.
This integration should be run on your local machine. Your API key and other information are
revealed to local users through the command line.
Depends on: google-api-python-client
""")
parser.add_option('--interval',
dest='interval',
default=30,
type=int,
action='store',
help='Minutes before event for reminder [default: 30]',
metavar='MINUTES')
parser.add_option('--calendar',
dest = 'calendarID',
default = 'primary',
type = str,
action = 'store',
help = 'Calendar ID for the calendar you want to receive reminders from.')
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
if not (options.zulip_email):
parser.error('You must specify --user')
zulip_client = zulip.init_from_options(options)
def get_credentials():
# type: () -> client.Credentials
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
an exception is thrown and the user is informed to run the script in this directory to get
credentials.
Returns:
Credentials, the obtained credential.
"""
try:
credential_path = os.path.join(HOME_DIR,
'google-credentials.json')
store = Storage(credential_path)
credentials = store.get()
return credentials
except client.Error:
logging.exception('Error while trying to open the `google-credentials.json` file.')
except IOError:
logging.error("Run the get-google-credentials script from this directory first.")
def populate_events():
# type: () -> Optional[None]
global events
credentials = get_credentials()
creds = credentials.authorize(httplib2.Http())
service = discovery.build('calendar', 'v3', http=creds)
now = datetime.datetime.now(pytz.utc).isoformat()
feed = service.events().list(calendarId=options.calendarID, timeMin=now, maxResults=5,
singleEvents=True, orderBy='startTime').execute()
events = []
for event in feed["items"]:
try:
start = dateutil.parser.parse(event["start"]["dateTime"])
# According to the API documentation, a time zone offset is required
# for start.dateTime unless a time zone is explicitly specified in
# start.timeZone.
if start.tzinfo is None:
event_timezone = pytz.timezone(event["start"]["timeZone"])
# pytz timezones include an extra localize method that's not part
# of the tzinfo base class.
start = event_timezone.localize(start) # type: ignore
except KeyError:
# All-day events can have only a date.
start_naive = dateutil.parser.parse(event["start"]["date"])
# All-day events don't have a time zone offset; instead, we use the
# time zone of the calendar.
calendar_timezone = pytz.timezone(feed["timeZone"])
# pytz timezones include an extra localize method that's not part
# of the tzinfo base class.
start = calendar_timezone.localize(start_naive) # type: ignore
try:
events.append((event["id"], start, event["summary"]))
except KeyError:
events.append((event["id"], start, "(No Title)"))
def send_reminders():
# type: () -> Optional[None]
global sent
messages = []
keys = set()
now = datetime.datetime.now(tz=pytz.utc)
for id, start, summary in events:
dt = start - now
if dt.days == 0 and dt.seconds < 60 * options.interval:
# The unique key includes the start time, because of
# repeating events.
key = (id, start)
if key not in sent:
if start.hour == 0 and start.minute == 0:
line = '%s is today.' % (summary,)
else:
line = '%s starts at %s' % (summary, start.strftime('%H:%M'))
print('Sending reminder:', line)
messages.append(line)
keys.add(key)
if not messages:
return
if len(messages) == 1:
message = 'Reminder: ' + messages[0]
else:
message = 'Reminder:\n\n' + '\n'.join('* ' + m for m in messages)
zulip_client.send_message(dict(
type = 'private',
to = options.zulip_email,
sender = options.zulip_email,
content = message))
sent.update(keys)
# Loop forever
for i in itertools.count():
try:
# We check reminders every minute, but only
# download the calendar every 10 minutes.
if not i % 10:
populate_events()
send_reminders()
except Exception:
logging.exception("Couldn't download Google calendar and/or couldn't post to Zulip.")
time.sleep(60)

179
zulip/integrations/hg/zulip-changegroup.py Executable file
View file

@ -0,0 +1,179 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Zulip hook for Mercurial changeset pushes.
# Copyright © 2012-2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#
#
# This hook is called when changesets are pushed to the master repository (ie
# `hg push`). See https://zulipchat.com/integrations for installation instructions.
from __future__ import absolute_import
import zulip
from six.moves import range
from typing import Any, Optional, Text
from mercurial import ui, repo
VERSION = "0.9"
def format_summary_line(web_url, user, base, tip, branch, node):
# type: (str, str, int, int, str, Text) -> Text
"""
Format the first line of the message, which contains summary
information about the changeset and links to the changelog if a
web URL has been configured:
Jane Doe <jane@example.com> pushed 1 commit to master (170:e494a5be3393):
"""
revcount = tip - base
plural = "s" if revcount > 1 else ""
if web_url:
shortlog_base_url = web_url.rstrip("/") + "/shortlog/"
summary_url = "{shortlog}{tip}?revcount={revcount}".format(
shortlog=shortlog_base_url, tip=tip - 1, revcount=revcount)
formatted_commit_count = "[{revcount} commit{s}]({url})".format(
revcount=revcount, s=plural, url=summary_url)
else:
formatted_commit_count = "{revcount} commit{s}".format(
revcount=revcount, s=plural)
return u"**{user}** pushed {commits} to **{branch}** (`{tip}:{node}`):\n\n".format(
user=user, commits=formatted_commit_count, branch=branch, tip=tip,
node=node[:12])
def format_commit_lines(web_url, repo, base, tip):
# type: (str, repo, int, int) -> str
"""
Format the per-commit information for the message, including the one-line
commit summary and a link to the diff if a web URL has been configured:
"""
if web_url:
rev_base_url = web_url.rstrip("/") + "/rev/"
commit_summaries = []
for rev in range(base, tip):
rev_node = repo.changelog.node(rev)
rev_ctx = repo.changectx(rev_node)
one_liner = rev_ctx.description().split("\n")[0]
if web_url:
summary_url = rev_base_url + str(rev_ctx)
summary = "* [{summary}]({url})".format(
summary=one_liner, url=summary_url)
else:
summary = "* {summary}".format(summary=one_liner)
commit_summaries.append(summary)
return "\n".join(summary for summary in commit_summaries)
def send_zulip(email, api_key, site, stream, subject, content):
# type: (str, str, str, str, str, Text) -> None
"""
Send a message to Zulip using the provided credentials, which should be for
a bot in most cases.
"""
client = zulip.Client(email=email, api_key=api_key,
site=site,
client="ZulipMercurial/" + VERSION)
message_data = {
"type": "stream",
"to": stream,
"subject": subject,
"content": content,
}
client.send_message(message_data)
def get_config(ui, item):
# type: (ui, str) -> Optional[str]
try:
# configlist returns everything in lists.
return ui.configlist('zulip', item)[0]
except IndexError:
return None
def hook(ui, repo, **kwargs):
# type: (ui, repo, **Text) -> None
"""
Invoked by configuring a [hook] entry in .hg/hgrc.
"""
hooktype = kwargs["hooktype"]
node = kwargs["node"]
ui.debug("Zulip: received {hooktype} event\n".format(hooktype=hooktype))
if hooktype != "changegroup":
ui.warn("Zulip: {hooktype} not supported\n".format(hooktype=hooktype))
exit(1)
ctx = repo.changectx(node)
branch = ctx.branch()
# If `branches` isn't specified, notify on all branches.
branch_whitelist = get_config(ui, "branches")
branch_blacklist = get_config(ui, "ignore_branches")
if branch_whitelist:
# Only send notifications on branches we are watching.
watched_branches = [b.lower().strip() for b in branch_whitelist.split(",")]
if branch.lower() not in watched_branches:
ui.debug("Zulip: ignoring event for {branch}\n".format(branch=branch))
exit(0)
if branch_blacklist:
# Don't send notifications for branches we've ignored.
ignored_branches = [b.lower().strip() for b in branch_blacklist.split(",")]
if branch.lower() in ignored_branches:
ui.debug("Zulip: ignoring event for {branch}\n".format(branch=branch))
exit(0)
# The first and final commits in the changeset.
base = repo[node].rev()
tip = len(repo)
email = get_config(ui, "email")
api_key = get_config(ui, "api_key")
site = get_config(ui, "site")
if not (email and api_key):
ui.warn("Zulip: missing email or api_key configurations\n")
ui.warn("in the [zulip] section of your .hg/hgrc.\n")
exit(1)
stream = get_config(ui, "stream")
# Give a default stream if one isn't provided.
if not stream:
stream = "commits"
web_url = get_config(ui, "web_url")
user = ctx.user()
content = format_summary_line(web_url, user, base, tip, branch, node)
content += format_commit_lines(web_url, repo, base, tip)
subject = branch
ui.debug("Sending to Zulip:\n")
ui.debug(content + "\n")
send_zulip(email, api_key, site, stream, subject, content)

142
zulip/integrations/irc/irc-mirror.py Executable file
View file

@ -0,0 +1,142 @@
#!/usr/bin/env python
#
# EXPERIMENTAL
# IRC <=> Zulip mirroring bot
#
# Setup: First, you need to install python-irc version 8.5.3
# (https://github.com/jaraco/irc)
from __future__ import print_function
import irc.bot
import irc.strings
from irc.client import ip_numstr_to_quad, ip_quad_to_numstr, Event, ServerConnection
import zulip
import optparse
if False:
from typing import Any, Dict
IRC_DOMAIN = "irc.example.com"
def zulip_sender(sender_string):
# type: (str) -> str
nick = sender_string.split("!")[0]
return nick + "@" + IRC_DOMAIN
class IRCBot(irc.bot.SingleServerIRCBot):
def __init__(self, channel, nickname, server, port=6667):
# type: (irc.bot.Channel, str, str, int) -> None
irc.bot.SingleServerIRCBot.__init__(self, [(server, port)], nickname, nickname)
self.channel = channel # type: irc.bot.Channel
def on_nicknameinuse(self, c, e):
# type: (ServerConnection, Event) -> None
c.nick(c.get_nickname().replace("_zulip", "__zulip"))
def on_welcome(self, c, e):
# type: (ServerConnection, Event) -> None
c.join(self.channel)
def forward_to_irc(msg):
# type: (Dict[str, Any]) -> None
if msg["type"] == "stream":
send = lambda x: c.privmsg(msg["display_recipient"], x)
else:
recipients = [u["short_name"] for u in msg["display_recipient"] if
u["email"] != msg["sender_email"]]
if len(recipients) == 1:
send = lambda x: c.privmsg(recipients[0], x)
else:
send = lambda x: c.privmsg_many(recipients, x)
for line in msg["content"].split("\n"):
send(line)
## Forwarding from Zulip => IRC is disabled; uncomment the next
## line to make this bot forward in that direction instead.
#
# zulip_client.call_on_each_message(forward_to_irc)
def on_privmsg(self, c, e):
# type: (ServerConnection, Event) -> None
content = e.arguments[0]
sender = zulip_sender(e.source)
if sender.endswith("_zulip@" + IRC_DOMAIN):
return
# Forward the PM to Zulip
print(zulip_client.send_message({
"sender": sender,
"type": "private",
"to": "username@example.com",
"content": content,
}))
def on_pubmsg(self, c, e):
# type: (ServerConnection, Event) -> None
content = e.arguments[0]
stream = e.target
sender = zulip_sender(e.source)
if sender.endswith("_zulip@" + IRC_DOMAIN):
return
# Forward the stream message to Zulip
print(zulip_client.send_message({
"forged": "yes",
"sender": sender,
"type": "stream",
"to": stream,
"subject": "IRC",
"content": content,
}))
def on_dccmsg(self, c, e):
# type: (ServerConnection, Event) -> None
c.privmsg("You said: " + e.arguments[0])
def on_dccchat(self, c, e):
# type: (ServerConnection, Event) -> None
if len(e.arguments) != 2:
return
args = e.arguments[1].split()
if len(args) == 4:
try:
address = ip_numstr_to_quad(args[2])
port = int(args[3])
except ValueError:
return
self.dcc_connect(address, port)
usage = """./irc-mirror.py --server=IRC_SERVER --channel=<CHANNEL> --nick-prefix=<NICK> [optional args]
Example:
./irc-mirror.py --irc-server=127.0.0.1 --channel='#test' --nick-prefix=username
--site=https://zulip.example.com --user=irc-bot@example.com
--api-key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Note that "_zulip" will be automatically appended to the IRC nick provided
Also note that at present you need to edit this code to do the Zulip => IRC side
"""
if __name__ == "__main__":
parser = optparse.OptionParser(usage=usage)
parser.add_option('--irc-server', default=None)
parser.add_option('--port', default=6667)
parser.add_option('--nick-prefix', default=None)
parser.add_option('--channel', default=None)
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
if options.irc_server is None or options.nick_prefix is None or options.channel is None:
parser.error("Missing required argument")
# Setting the client to irc_mirror is critical for this to work
options.client = "irc_mirror"
zulip_client = zulip.init_from_options(options)
nickname = options.nick_prefix + "_zulip"
bot = IRCBot(options.channel, nickname, options.irc_server, options.port)
bot.start()

63
zulip/integrations/jabber/jabber_mirror.py Executable file
View file

@ -0,0 +1,63 @@
#!/usr/bin/env python
# Copyright (C) 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from __future__ import print_function
import sys
import subprocess
import os
import traceback
import signal
from types import FrameType
from typing import Any
from zulip import RandomExponentialBackoff
def die(signal, frame):
# type: (int, FrameType) -> None
"""We actually want to exit, so run os._exit (so as not to be caught and restarted)"""
os._exit(1)
signal.signal(signal.SIGINT, die)
args = [os.path.join(os.path.dirname(sys.argv[0]), "jabber_mirror_backend.py")]
args.extend(sys.argv[1:])
backoff = RandomExponentialBackoff(timeout_success_equivalent=300)
while backoff.keep_going():
print("Starting Jabber mirroring bot")
try:
ret = subprocess.call(args)
except Exception:
traceback.print_exc()
else:
if ret == 2:
# Don't try again on initial configuration errors
sys.exit(ret)
backoff.fail()
print("")
print("")
print("ERROR: The Jabber mirroring bot is unable to continue mirroring Jabber.")
print("Please contact zulip-devel@googlegroups.com if you need assistance.")
print("")
sys.exit(1)

477
zulip/integrations/jabber/jabber_mirror_backend.py Executable file
View file

@ -0,0 +1,477 @@
#!/usr/bin/env python
#
# Copyright (C) 2013 Permabit, Inc.
# Copyright (C) 2013--2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
# The following is a table showing which kinds of messages are handled by the
# mirror in each mode:
#
# Message origin/type --> | Jabber | Zulip
# Mode/sender-, +-----+----+--------+----
# V | MUC | PM | stream | PM
# --------------+-------------+-----+----+--------+----
# | other sender| | x | |
# personal mode +-------------+-----+----+--------+----
# | self sender | | x | x | x
# ------------- +-------------+-----+----+--------+----
# | other sender| x | | |
# public mode +-------------+-----+----+--------+----
# | self sender | | | |
from typing import Dict, List, Set
import logging
import threading
import optparse
from sleekxmpp import ClientXMPP, InvalidJID, JID
from sleekxmpp.stanza import Message as JabberMessage
from sleekxmpp.exceptions import IqError, IqTimeout
from six.moves.configparser import SafeConfigParser
import getpass
import os
import sys
import zulip
from zulip import Client
import re
from typing import Any, Callable
__version__ = "1.1"
def room_to_stream(room):
# type: (str) -> str
return room + "/xmpp"
def stream_to_room(stream):
# type: (str) -> str
return stream.lower().rpartition("/xmpp")[0]
def jid_to_zulip(jid):
# type: (JID) -> str
suffix = ''
if not jid.username.endswith("-bot"):
suffix = options.zulip_email_suffix
return "%s%s@%s" % (jid.username, suffix, options.zulip_domain)
def zulip_to_jid(email, jabber_domain):
# type: (str, str) -> JID
jid = JID(email, domain=jabber_domain)
if (options.zulip_email_suffix and
options.zulip_email_suffix in jid.username and
not jid.username.endswith("-bot")):
jid.username = jid.username.rpartition(options.zulip_email_suffix)[0]
return jid
class JabberToZulipBot(ClientXMPP):
def __init__(self, jid, password, rooms):
# type: (JID, str, List[str]) -> None
if jid.resource:
self.nick = jid.resource
else:
self.nick = jid.username
jid.resource = "zulip"
ClientXMPP.__init__(self, jid, password)
self.rooms = set() # type: Set[str]
self.rooms_to_join = rooms
self.add_event_handler("session_start", self.session_start)
self.add_event_handler("message", self.message)
self.zulip = None # type: Client
self.use_ipv6 = False
self.register_plugin('xep_0045') # Jabber chatrooms
self.register_plugin('xep_0199') # XMPP Ping
def set_zulip_client(self, zulipToJabberClient):
# type: (ZulipToJabberBot) -> None
self.zulipToJabber = zulipToJabberClient
def session_start(self, event):
# type: (Dict[str, Any]) -> None
self.get_roster()
self.send_presence()
for room in self.rooms_to_join:
self.join_muc(room)
def join_muc(self, room):
# type: (str) -> None
if room in self.rooms:
return
logging.debug("Joining " + room)
self.rooms.add(room)
muc_jid = JID(local=room, domain=options.conference_domain)
xep0045 = self.plugin['xep_0045']
try:
xep0045.joinMUC(muc_jid, self.nick, wait=True)
except InvalidJID:
logging.error("Could not join room: " + str(muc_jid))
return
# Configure the room. Really, we should only do this if the room is
# newly created.
form = None
try:
form = xep0045.getRoomConfig(muc_jid)
except ValueError:
pass
if form:
xep0045.configureRoom(muc_jid, form)
else:
logging.error("Could not configure room: " + str(muc_jid))
def leave_muc(self, room):
# type: (str) -> None
if room not in self.rooms:
return
logging.debug("Leaving " + room)
self.rooms.remove(room)
muc_jid = JID(local=room, domain=options.conference_domain)
self.plugin['xep_0045'].leaveMUC(muc_jid, self.nick)
def message(self, msg):
# type: (JabberMessage) -> Any
try:
if msg["type"] == "groupchat":
return self.group(msg)
elif msg["type"] == "chat":
return self.private(msg)
else:
logging.warning("Got unexpected message type")
logging.warning(msg)
except Exception:
logging.exception("Error forwarding Jabber => Zulip")
def private(self, msg):
# type: (JabberMessage) -> None
if options.mode == 'public' or msg['thread'] == u'\u1FFFE':
return
sender = jid_to_zulip(msg["from"])
recipient = jid_to_zulip(msg["to"])
zulip_message = dict(
sender = sender,
type = "private",
to = recipient,
content = msg["body"],
)
ret = self.zulipToJabber.client.send_message(zulip_message)
if ret.get("result") != "success":
logging.error(str(ret))
def group(self, msg):
# type: (JabberMessage) -> None
if options.mode == 'personal' or msg["thread"] == u'\u1FFFE':
return
subject = msg["subject"]
if len(subject) == 0:
subject = "(no topic)"
stream = room_to_stream(msg['from'].local)
sender_nick = msg.get_mucnick()
if not sender_nick:
# Messages from the room itself have no nickname. We should not try
# to mirror these
return
jid = self.nickname_to_jid(msg.get_mucroom(), sender_nick)
sender = jid_to_zulip(jid)
zulip_message = dict(
forged = "yes",
sender = sender,
type = "stream",
subject = subject,
to = stream,
content = msg["body"],
)
ret = self.zulipToJabber.client.send_message(zulip_message)
if ret.get("result") != "success":
logging.error(str(ret))
def nickname_to_jid(self, room, nick):
# type: (str, str) -> JID
jid = self.plugin['xep_0045'].getJidProperty(room, nick, "jid")
if (jid is None or jid == ''):
return JID(local=nick.replace(' ', ''), domain=self.boundjid.domain)
else:
return jid
class ZulipToJabberBot(object):
def __init__(self, zulip_client):
# type: (Client) -> None
self.client = zulip_client
self.jabber = None # type: JabberToZulipBot
def set_jabber_client(self, client):
# type: (JabberToZulipBot) -> None
self.jabber = client
def process_event(self, event):
# type: (Dict[str, Any]) -> None
if event['type'] == 'message':
message = event["message"]
if message['sender_email'] != self.client.email:
return
try:
if message['type'] == 'stream':
self.stream_message(message)
elif message['type'] == 'private':
self.private_message(message)
except Exception:
logging.exception("Exception forwarding Zulip => Jabber")
elif event['type'] == 'subscription':
self.process_subscription(event)
elif event['type'] == 'stream':
self.process_stream(event)
def stream_message(self, msg):
# type: (Dict[str, str]) -> None
stream = msg['display_recipient']
if not stream.endswith("/xmpp"):
return
room = stream_to_room(stream)
jabber_recipient = JID(local=room, domain=options.conference_domain)
outgoing = self.jabber.make_message(
mto = jabber_recipient,
mbody = msg['content'],
mtype = 'groupchat')
outgoing['thread'] = u'\u1FFFE'
outgoing.send()
def private_message(self, msg):
# type: (Dict[str, Any]) -> None
for recipient in msg['display_recipient']:
if recipient["email"] == self.client.email:
continue
if not recipient["is_mirror_dummy"]:
continue
recip_email = recipient['email']
jabber_recipient = zulip_to_jid(recip_email, self.jabber.boundjid.domain)
outgoing = self.jabber.make_message(
mto = jabber_recipient,
mbody = msg['content'],
mtype = 'chat')
outgoing['thread'] = u'\u1FFFE'
outgoing.send()
def process_subscription(self, event):
# type: (Dict[str, Any]) -> None
if event['op'] == 'add':
streams = [s['name'].lower() for s in event['subscriptions']]
streams = [s for s in streams if s.endswith("/xmpp")]
for stream in streams:
self.jabber.join_muc(stream_to_room(stream))
if event['op'] == 'remove':
streams = [s['name'].lower() for s in event['subscriptions']]
streams = [s for s in streams if s.endswith("/xmpp")]
for stream in streams:
self.jabber.leave_muc(stream_to_room(stream))
def process_stream(self, event):
# type: (Dict[str, Any]) -> None
if event['op'] == 'occupy':
streams = [s['name'].lower() for s in event['streams']]
streams = [s for s in streams if s.endswith("/xmpp")]
for stream in streams:
self.jabber.join_muc(stream_to_room(stream))
if event['op'] == 'vacate':
streams = [s['name'].lower() for s in event['streams']]
streams = [s for s in streams if s.endswith("/xmpp")]
for stream in streams:
self.jabber.leave_muc(stream_to_room(stream))
def get_rooms(zulipToJabber):
# type: (ZulipToJabberBot) -> List[str]
def get_stream_infos(key, method):
# type: (str, Callable) -> Any
ret = method()
if ret.get("result") != "success":
logging.error(ret)
sys.exit("Could not get initial list of Zulip %s" % (key,))
return ret[key]
if options.mode == 'public':
stream_infos = get_stream_infos("streams", zulipToJabber.client.get_streams)
else:
stream_infos = get_stream_infos("subscriptions", zulipToJabber.client.list_subscriptions)
rooms = [] # type: List[str]
for stream_info in stream_infos:
stream = stream_info['name']
if stream.endswith("/xmpp"):
rooms.append(stream_to_room(stream))
return rooms
def config_error(msg):
# type: (str) -> None
sys.stderr.write("%s\n" % (msg,))
sys.exit(2)
if __name__ == '__main__':
parser = optparse.OptionParser(
epilog='''Most general and Jabber configuration options may also be specified in the
zulip configuration file under the jabber_mirror section (exceptions are noted
in their help sections). Keys have the same name as options with hyphens
replaced with underscores. Zulip configuration options go in the api section,
as normal.'''.replace("\n", " ")
)
parser.add_option(
'--mode',
default=None,
action='store',
help='''Which mode to run in. Valid options are "personal" and "public". In
"personal" mode, the mirror uses an individual users' credentials and mirrors
all messages they send on Zulip to Jabber and all private Jabber messages to
Zulip. In "public" mode, the mirror uses the credentials for a dedicated mirror
user and mirrors messages sent to Jabber rooms to Zulip. Defaults to
"personal"'''.replace("\n", " "))
parser.add_option(
'--zulip-email-suffix',
default=None,
action='store',
help='''Add the specified suffix to the local part of email addresses constructed
from JIDs and nicks before sending requests to the Zulip server, and remove the
suffix before sending requests to the Jabber server. For example, specifying
"+foo" will cause messages that are sent to the "bar" room by nickname "qux" to
be mirrored to the "bar/xmpp" stream in Zulip by user "qux+foo@example.com". This
option does not affect login credentials.'''.replace("\n", " "))
parser.add_option('-d', '--debug',
help='set logging to DEBUG. Can not be set via config file.',
action='store_const',
dest='log_level',
const=logging.DEBUG,
default=logging.INFO)
jabber_group = optparse.OptionGroup(parser, "Jabber configuration") # type: ignore # https://github.com/python/typeshed/pull/1248
jabber_group.add_option(
'--jid',
default=None,
action='store',
help="Your Jabber JID. If a resource is specified, "
"it will be used as the nickname when joining MUCs. "
"Specifying the nickname is mostly useful if you want "
"to run the public mirror from a regular user instead of "
"from a dedicated account.")
jabber_group.add_option('--jabber-password',
default=None,
action='store',
help="Your Jabber password")
jabber_group.add_option('--conference-domain',
default=None,
action='store',
help="Your Jabber conference domain (E.g. conference.jabber.example.com). "
"If not specifed, \"conference.\" will be prepended to your JID's domain.")
jabber_group.add_option('--no-use-tls',
default=None,
action='store_true')
jabber_group.add_option('--jabber-server-address',
default=None,
action='store',
help="The hostname of your Jabber server. This is only needed if "
"your server is missing SRV records")
jabber_group.add_option('--jabber-server-port',
default='5222',
action='store',
help="The port of your Jabber server. This is only needed if "
"your server is missing SRV records")
parser.add_option_group(jabber_group)
parser.add_option_group(zulip.generate_option_group(parser, "zulip-"))
(options, args) = parser.parse_args()
logging.basicConfig(level=options.log_level,
format='%(levelname)-8s %(message)s')
if options.zulip_config_file is None:
config_file = zulip.get_default_config_filename()
else:
config_file = options.zulip_config_file
config = SafeConfigParser()
try:
with open(config_file, 'r') as f:
config.readfp(f, config_file)
except IOError:
pass
for option in ("jid", "jabber_password", "conference_domain", "mode", "zulip_email_suffix",
"jabber_server_address", "jabber_server_port"):
if (getattr(options, option) is None and
config.has_option("jabber_mirror", option)):
setattr(options, option, config.get("jabber_mirror", option))
for option in ("no_use_tls",):
if getattr(options, option) is None:
if config.has_option("jabber_mirror", option):
setattr(options, option, config.getboolean("jabber_mirror", option))
else:
setattr(options, option, False)
if options.mode is None:
options.mode = "personal"
if options.zulip_email_suffix is None:
options.zulip_email_suffix = ''
if options.mode not in ('public', 'personal'):
config_error("Bad value for --mode: must be one of 'public' or 'personal'")
if None in (options.jid, options.jabber_password):
config_error("You must specify your Jabber JID and Jabber password either "
"in the Zulip configuration file or on the commandline")
zulipToJabber = ZulipToJabberBot(zulip.init_from_options(options, "JabberMirror/" + __version__))
# This won't work for open realms that don't have a consistent domain
options.zulip_domain = zulipToJabber.client.email.partition('@')[-1]
try:
jid = JID(options.jid)
except InvalidJID as e:
config_error("Bad JID: %s: %s" % (options.jid, e.message))
if options.conference_domain is None:
options.conference_domain = "conference.%s" % (jid.domain,)
xmpp = JabberToZulipBot(jid, options.jabber_password, get_rooms(zulipToJabber))
address = None
if options.jabber_server_address:
address = (options.jabber_server_address, options.jabber_server_port)
if not xmpp.connect(use_tls=not options.no_use_tls, address=address):
sys.exit("Unable to connect to Jabber server")
xmpp.set_zulip_client(zulipToJabber)
zulipToJabber.set_jabber_client(xmpp)
xmpp.process(block=False)
if options.mode == 'public':
event_types = ['stream']
else:
event_types = ['message', 'subscription']
try:
logging.info("Connecting to Zulip.")
zulipToJabber.client.call_on_each_event(zulipToJabber.process_event,
event_types=event_types)
except BaseException as e:
logging.exception("Exception in main loop")
xmpp.abort()
sys.exit(1)

149
zulip/integrations/jira/org/humbug/jira/ZulipListener.groovy Normal file
View file

@ -0,0 +1,149 @@
/*
* Copyright (c) 2014 Zulip, Inc
*/
package org.zulip.jira
import static com.atlassian.jira.event.type.EventType.*
import com.atlassian.jira.event.issue.AbstractIssueEventListener
import com.atlassian.jira.event.issue.IssueEvent
import java.util.logging.Level
import java.util.logging.Logger
import org.apache.commons.httpclient.HttpClient
import org.apache.commons.httpclient.HttpStatus;
import org.apache.commons.httpclient.methods.PostMethod
import org.apache.commons.httpclient.NameValuePair
class ZulipListener extends AbstractIssueEventListener {
Logger LOGGER = Logger.getLogger(ZulipListener.class.getName());
// The email address of one of the bots you created on your Zulip settings page.
String zulipEmail = ""
// That bot's API key.
String zulipAPIKey = ""
// What stream to send messages to. Must already exist.
String zulipStream = "jira"
// The base JIRA url for browsing
String issueBaseUrl = "https://jira.COMPANY.com/browse/"
// Your zulip domain
String base_url = "https://zulip.example.com/"
@Override
void workflowEvent(IssueEvent event) {
processIssueEvent(event)
}
String processIssueEvent(IssueEvent event) {
String author = event.user.displayName
String issueId = event.issue.key
String issueUrl = issueBaseUrl + issueId
String issueUrlMd = String.format("[%s](%s)", issueId, issueBaseUrl + issueId)
String title = event.issue.summary
String subject = truncate(String.format("%s: %s", issueId, title), 60)
String assignee = "no one"
if (event.issue.assignee) {
assignee = event.issue.assignee.name
}
String comment = "";
if (event.comment) {
comment = event.comment.body
}
String content;
// Event types:
// https://docs.atlassian.com/jira/5.0/com/atlassian/jira/event/type/EventType.html
// Issue API:
// https://docs.atlassian.com/jira/5.0/com/atlassian/jira/issue/Issue.html
switch (event.getEventTypeId()) {
case ISSUE_COMMENTED_ID:
content = String.format("%s **updated** %s with comment:\n\n> %s",
author, issueUrlMd, comment)
break
case ISSUE_CREATED_ID:
content = String.format("%s **created** %s priority %s, assigned to @**%s**: \n\n> %s",
author, issueUrlMd, event.issue.priorityObject.name,
assignee, title)
break
case ISSUE_ASSIGNED_ID:
content = String.format("%s **reassigned** %s to **%s**",
author, issueUrlMd, assignee)
break
case ISSUE_DELETED_ID:
content = String.format("%s **deleted** %s!",
author, issueUrlMd)
break
case ISSUE_RESOLVED_ID:
content = String.format("%s **resolved** %s as %s:\n\n> %s",
author, issueUrlMd, event.issue.resolutionObject.name,
comment)
break
case ISSUE_CLOSED_ID:
content = String.format("%s **closed** %s with resolution %s:\n\n> %s",
author, issueUrlMd, event.issue.resolutionObject.name,
comment)
break
case ISSUE_REOPENED_ID:
content = String.format("%s **reopened** %s:\n\n> %s",
author, issueUrlMd, comment)
break
default:
return
}
sendStreamMessage(zulipStream, subject, content)
}
String post(String method, NameValuePair[] parameters) {
PostMethod post = new PostMethod(zulipUrl(method))
post.setRequestHeader("Content-Type", post.FORM_URL_ENCODED_CONTENT_TYPE)
// TODO: Include more useful data in the User-agent
post.setRequestHeader("User-agent", "ZulipJira/0.1")
try {
post.setRequestBody(parameters)
HttpClient client = new HttpClient()
client.executeMethod(post)
String response = post.getResponseBodyAsString()
if (post.getStatusCode() != HttpStatus.SC_OK) {
String params = ""
for (NameValuePair pair: parameters) {
params += "\n" + pair.getName() + ":" + pair.getValue()
}
LOGGER.log(Level.SEVERE, "Error sending Zulip message:\n" + response + "\n\n" +
"We sent:" + params)
}
return response;
} catch (IOException e) {
throw new RuntimeException(e)
} finally {
post.releaseConnection()
}
}
String truncate(String string, int length) {
if (string.length() < length) {
return string
}
return string.substring(0, length - 3) + "..."
}
String sendStreamMessage(String stream, String subject, String message) {
NameValuePair[] body = [new NameValuePair("api-key", zulipAPIKey),
new NameValuePair("email", zulipEmail),
new NameValuePair("type", "stream"),
new NameValuePair("to", stream),
new NameValuePair("subject", subject),
new NameValuePair("content", message)]
return post("send_message", body);
}
String zulipUrl(method) {
return base_url.replaceAll("/+$", "") + "/api/v1/" + method
}
}

117
zulip/integrations/log2zulip/log2zulip Executable file
View file

@ -0,0 +1,117 @@
#!/usr/bin/env python
from __future__ import print_function
import errno
import os
import platform
import re
import sys
import shutil
import subprocess
import traceback
try:
# Use the Zulip virtualenv if available
sys.path.append(os.path.join(os.path.dirname(__file__), "../../.."))
import scripts.lib.setup_path_on_import
except ImportError:
pass
import json
import ujson
sys.path.insert(0, os.path.join(os.path.dirname(__file__), "../../"))
import zulip
from typing import List
lock_path = "/var/tmp/log2zulip.lock"
control_path = "/etc/log2zulip.conf"
def mkdir_p(path):
# type: (str) -> None
# Python doesn't have an analog to `mkdir -p` < Python 3.2.
try:
os.makedirs(path)
except OSError as e:
if e.errno == errno.EEXIST and os.path.isdir(path):
pass
else:
raise
def send_log_zulip(file_name, count, lines, extra=""):
# type: (str, int, List[str], str) -> None
content = "%s new errors%s:\n```\n%s\n```" % (count, extra, "\n".join(lines))
zulip_client.send_message({
"type": "stream",
"to": "logs",
"subject": "%s on %s" % (file_name, platform.node()),
"content": content,
})
def process_lines(raw_lines, file_name):
# type: (List[str], str) -> None
lines = []
for line in raw_lines:
# Add any filtering or modification code here
if re.match(".*upstream timed out.*while reading upstream.*", line):
continue
lines.append(line)
if len(lines) == 0:
return
elif len(lines) > 10:
send_log_zulip(file_name, len(lines), lines[0:3], extra=", examples include")
else:
send_log_zulip(file_name, len(lines), lines)
def process_logs():
# type: () -> None
for filename in log_files:
data_file_path = "/var/tmp/log2zulip.state"
mkdir_p(os.path.dirname(data_file_path))
if not os.path.exists(data_file_path):
open(data_file_path, "w").write("{}")
last_data = ujson.loads(open(data_file_path).read())
new_data = {}
for log_file in log_files:
file_data = last_data.get(log_file, {})
if not os.path.exists(log_file):
# If the file doesn't exist, log an error and then move on to the next file
print("Log file does not exist or could not stat log file: %s" % (log_file,))
continue
length = int(subprocess.check_output(["wc", "-l", log_file]).split()[0])
if file_data.get("last") is None:
file_data["last"] = 1
if length + 1 < file_data["last"]:
# The log file was rotated, restart from empty. Note that
# because we don't actually store the log file content, if
# a log file ends up at the same line length as before
# immediately after rotation, this tool won't notice.
file_data["last"] = 1
new_lines = subprocess.check_output(["tail", "-n+%s" % (file_data["last"],), log_file]).split('\n')[:-1]
if len(new_lines) > 0:
process_lines(new_lines, filename)
file_data["last"] += len(new_lines)
new_data[log_file] = file_data
open(data_file_path, "w").write(ujson.dumps(new_data))
if __name__ == "__main__":
if os.path.exists(lock_path):
print("Log2zulip lock held; not doing anything")
sys.exit(0)
try:
open(lock_path, "w").write("1")
zulip_client = zulip.Client(config_file="/etc/log2zulip.zuliprc")
try:
log_files = ujson.loads(open(control_path, "r").read())
except Exception:
print("Could not load control data from %s" % (control_path,))
traceback.print_exc()
sys.exit(1)
process_logs()
finally:
try:
os.remove(lock_path)
except OSError as IOError:
pass

52
zulip/integrations/nagios/nagios-notify-zulip Executable file
View file

@ -0,0 +1,52 @@
#!/usr/bin/env python
import optparse
import zulip
from typing import List, Text, Dict, Any
VERSION = "0.9"
# Nagios passes the notification details as command line options.
# In Nagios, "output" means "first line of output", and "long
# output" means "other lines of output".
parser = optparse.OptionParser() # type: optparse.OptionParser
parser.add_option('--output', default='')
parser.add_option('--long-output', default='')
parser.add_option('--stream', default='nagios')
parser.add_option('--config', default='/etc/nagios3/zuliprc')
for opt in ('type', 'host', 'service', 'state'):
parser.add_option('--' + opt)
(opts, args) = parser.parse_args() # type: Any, List[Text]
client = zulip.Client(config_file=opts.config,
client="ZulipNagios/" + VERSION) # type: zulip.Client
msg = dict(type='stream', to=opts.stream) # type: Dict[str, Any]
# Set a subject based on the host or service in question. This enables
# threaded discussion of multiple concurrent issues, and provides useful
# context when narrowed.
#
# We send PROBLEM and RECOVERY messages to the same subject.
if opts.service is None:
# Host notification
thing = 'host' # type: Text
msg['subject'] = 'host %s' % (opts.host,)
else:
# Service notification
thing = 'service'
msg['subject'] = 'service %s on %s' % (opts.service, opts.host)
if len(msg['subject']) > 60:
msg['subject'] = msg['subject'][0:57].rstrip() + "..."
# e.g. **PROBLEM**: service is CRITICAL
msg['content'] = '**%s**: %s is %s' % (opts.type, thing, opts.state)
# The "long output" can contain newlines represented by "\n" escape sequences.
# The Nagios mail command uses /usr/bin/printf "%b" to expand these.
# We will be more conservative and handle just this one escape sequence.
output = (opts.output + '\n' + opts.long_output.replace(r'\n', '\n')).strip() # type: Text
if output:
# Put any command output in a code block.
msg['content'] += ('\n\n~~~~\n' + output + "\n~~~~\n")
client.send_message(msg)

21
zulip/integrations/nagios/zulip_nagios.cfg Normal file
View file

@ -0,0 +1,21 @@
define contact{
contact_name zulip
alias zulip
service_notification_period 24x7
host_notification_period 24x7
service_notification_options w,u,c,r
host_notification_options d,r
service_notification_commands notify-service-by-zulip
host_notification_commands notify-host-by-zulip
}
# Zulip commands
define command {
command_name notify-host-by-zulip
command_line /usr/local/share/zulip/integrations/nagios/nagios-notify-zulip --stream=nagios --type="$NOTIFICATIONTYPE$" --host="$HOSTADDRESS$" --state="$HOSTSTATE$" --output="$HOSTOUTPUT$" --long-output="$LONGHOSTOUTPUT$"
}
define command {
command_name notify-service-by-zulip
command_line /usr/local/share/zulip/integrations/nagios/nagios-notify-zulip --stream=nagios --type="$NOTIFICATIONTYPE$" --host="$HOSTADDRESS$" --service="$SERVICEDESC$" --state="$SERVICESTATE$" --output="$SERVICEOUTPUT$" --long-output="$LONGSERVICEOUTPUT$"
}

5
zulip/integrations/nagios/zuliprc.example Normal file
View file

@ -0,0 +1,5 @@
# Fill these values in with the appropriate values for your realm, and
# then install this value at /etc/nagios3/zuliprc
[api]
email = nagios-bot@example.com
key = 0123456789abcdef0123456789abcdef

75
zulip/integrations/openshift/post_deploy Executable file
View file

@ -0,0 +1,75 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Zulip notification post-receive hook.
# Copyright © 2012-2017 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
import os
import subprocess
import sys
from typing import Dict
sys.path.insert(0, os.path.dirname(__file__))
import zulip_openshift_config as config
VERSION = '0.1'
if config.ZULIP_API_PATH is not None:
sys.path.append(config.ZULIP_API_PATH)
import zulip
client = zulip.Client(
email=config.ZULIP_USER,
site=config.ZULIP_SITE,
api_key=config.ZULIP_API_KEY,
client='ZulipOpenShift/' + VERSION)
def get_deployment_details():
# type: () -> Dict[str, str]
# "gear deployments" output example:
# Activation time - Deployment ID - Git Ref - Git SHA1
# 2017-01-07 15:40:30 -0500 - 9e2b7143 - master - b9ce57c - ACTIVE
dep = subprocess.check_output(['gear', 'deployments']).splitlines()[1]
splits = dep.split(' - ')
return dict(app_name=os.environ['OPENSHIFT_APP_NAME'],
url=os.environ['OPENSHIFT_APP_DNS'],
branch=splits[2],
commit_id=splits[3])
def send_bot_message(deployment):
# type: (Dict[str, str]) -> None
destination = config.deployment_notice_destination(deployment['branch'])
if destination is None:
# No message should be sent
return
message = config.format_deployment_message(**deployment)
client.send_message({
'type': 'stream',
'to': destination['stream'],
'subject': destination['subject'],
'content': message,
})
return
deployment = get_deployment_details()
send_bot_message(deployment)

75
zulip/integrations/openshift/zulip_openshift_config.py Executable file
View file

@ -0,0 +1,75 @@
# -*- coding: utf-8 -*-
#
# Copyright © 2017 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# https://github.com/python/mypy/issues/1141
from typing import Dict, Text
# Change these values to configure authentication for the plugin
ZULIP_USER = 'openshift-bot@example.com'
ZULIP_API_KEY = '0123456789abcdef0123456789abcdef'
# deployment_notice_destination() lets you customize where deployment notices
# are sent to with the full power of a Python function.
#
# It takes the following arguments:
# * branch = the name of the branch where the deployed commit was
# pushed to
#
# Returns a dictionary encoding the stream and subject to send the
# notification to (or None to send no notification).
#
# The default code below will send every commit pushed to "master" to
# * stream "deployments"
# * topic "master"
# And similarly for branch "test-post-receive" (for use when testing).
def deployment_notice_destination(branch):
# type: (str) -> Dict[str, Text]
if branch in ['master', 'test-post-receive']:
return dict(stream = 'deployments',
subject = u'%s' % (branch,))
# Return None for cases where you don't want a notice sent
return None
# Modify this function to change how deployments are displayed
#
# It takes the following arguments:
# * app_name = the name of the app being deployed
# * url = the FQDN (Fully Qualified Domain Name) where the app
# can be found
# * branch = the name of the branch where the deployed commit was
# pushed to
# * commit_id = hash of the commit that triggered the deployment
# * dep_id = deployment id
# * dep_time = deployment timestamp
def format_deployment_message(
app_name='', url='', branch='', commit_id='', dep_id='', dep_time=''):
# type: (str, str, str, str, str, str) -> str
return 'Deployed commit `%s` (%s) in [%s](%s)' % (
commit_id, branch, app_name, url)
## If properly installed, the Zulip API should be in your import
## path, but if not, set a custom path below
ZULIP_API_PATH = None # type: str
# Set this to your Zulip server's API URI
ZULIP_SITE = 'https://zulip.example.com'

3269
zulip/integrations/perforce/git_p4.py Executable file
View file

File diff suppressed because it is too large Load diff

26
zulip/integrations/perforce/license.txt Normal file
View file

@ -0,0 +1,26 @@
git_p4.py was downloaded from https://raw.github.com/git/git/34022ba/git-p4.py
The header of that file references <http://opensource.org/licenses/mit-license.php>,
the textual contents of which are included below.
The MIT License (MIT)
Copyright (c) <year> <copyright holders>
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

117
zulip/integrations/perforce/zulip_change-commit.py Executable file
View file

@ -0,0 +1,117 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Copyright © 2012-2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
'''Zulip notification change-commit hook.
In Perforce, The "change-commit" trigger is fired after a metadata has been
created, files have been transferred, and the changelist committed to the depot
database.
This specific trigger expects command-line arguments in the form:
%change% %changeroot%
For example:
1234 //depot/security/src/
'''
from __future__ import print_function
import os
import sys
import os.path
import git_p4
__version__ = "0.1"
sys.path.insert(0, os.path.dirname(__file__))
from typing import Any, Dict, Optional, Text
import zulip_perforce_config as config
if config.ZULIP_API_PATH is not None:
sys.path.append(config.ZULIP_API_PATH)
import zulip
client = zulip.Client(
email=config.ZULIP_USER,
site=config.ZULIP_SITE,
api_key=config.ZULIP_API_KEY,
client="ZulipPerforce/" + __version__) # type: zulip.Client
try:
changelist = int(sys.argv[1]) # type: int
changeroot = sys.argv[2] # type: str
except IndexError:
print("Wrong number of arguments.\n\n", end=' ', file=sys.stderr)
print(__doc__, file=sys.stderr)
sys.exit(-1)
except ValueError:
print("First argument must be an integer.\n\n", end=' ', file=sys.stderr)
print(__doc__, file=sys.stderr)
sys.exit(-1)
metadata = git_p4.p4_describe(changelist) # type: Dict[str, str]
destination = config.commit_notice_destination(changeroot, changelist) # type: Optional[Dict[str, str]]
if destination is None:
# Don't forward the notice anywhere
sys.exit(0)
ignore_missing_stream = None
if hasattr(config, "ZULIP_IGNORE_MISSING_STREAM"):
ignore_missing_stream = config.ZULIP_IGNORE_MISSING_STREAM
if ignore_missing_stream:
# Check if the destination stream exists yet
stream_state = client.get_stream_id(destination["stream"])
if stream_state["result"] == "error":
# Silently discard the message
sys.exit(0)
change = metadata["change"]
p4web = None
if hasattr(config, "P4_WEB"):
p4web = config.P4_WEB
if p4web is not None:
# linkify the change number
change = '[{change}]({p4web}/{change}?ac=10)'.format(p4web=p4web, change=change)
message = """**{user}** committed revision @{change} to `{path}`.
```quote
{desc}
```
""".format(
user=metadata["user"],
change=change,
path=changeroot,
desc=metadata["desc"]) # type: str
message_data = {
"type": "stream",
"to": destination["stream"],
"subject": destination["subject"],
"content": message,
} # type: Dict[str, Any]
client.send_message(message_data)

69
zulip/integrations/perforce/zulip_perforce_config.py Normal file
View file

@ -0,0 +1,69 @@
# -*- coding: utf-8 -*-
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# Change these values to configure authentication for the plugin
ZULIP_USER = "p4-bot@example.com"
ZULIP_API_KEY = "0123456789abcdef0123456789abcdef"
ZULIP_SITE = "https://zulip.example.com"
# Set this to True to silently drop messages if the destination stream
# does not exist. This prevents the warnings from Zulip's Notification Bot
# when commits are made on a branch for which no stream has been created.
ZULIP_IGNORE_MISSING_STREAM = False
# Set this to point at a p4web installation to get changelist IDs as links
# P4_WEB = "https://p4web.example.com"
P4_WEB = None
# commit_notice_destination() lets you customize where commit notices
# are sent to with the full power of a Python function.
#
# It takes the following arguments:
# * path = the path to the Perforce depot on the server
# * changelist = the changelist id
#
# Returns a dictionary encoding the stream and topic to send the
# notification to (or None to send no notification).
#
# The default code below will send every commit except for ones in the
# "master-plan" and "secret" subdirectories of //depot/ to:
# * stream "depot_subdirectory-commits"
# * subject "change_root"
def commit_notice_destination(path, changelist):
dirs = path.split('/')
if len(dirs) >= 4 and dirs[3] not in ("*", "..."):
directory = dirs[3]
else:
# No subdirectory, so just use "depot"
directory = dirs[2]
if directory not in ["evil-master-plan", "my-super-secret-repository"]:
return dict(stream = "%s-commits" % (directory,),
subject = path)
# Return None for cases where you don't want a notice sent
return None
## If properly installed, the Zulip API should be in your import
## path, but if not, set a custom path below
ZULIP_API_PATH = None

254
zulip/integrations/rss/rss-bot Normal file
View file

@ -0,0 +1,254 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# RSS integration for Zulip
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import calendar
import errno
import hashlib
from six.moves.html_parser import HTMLParser
import logging
import optparse
import os
import re
import sys
import time
from six.moves import urllib
from typing import Dict, List, Tuple, Any
import feedparser
import zulip
VERSION = "0.9" # type: str
RSS_DATA_DIR = os.path.expanduser(os.path.join('~', '.cache', 'zulip-rss')) # type: str
OLDNESS_THRESHOLD = 30 # type: int
usage = """Usage: Send summaries of RSS entries for your favorite feeds to Zulip.
This bot requires the feedparser module.
To use this script:
1. Create an RSS feed file containing 1 feed URL per line (default feed
file location: ~/.cache/zulip-rss/rss-feeds)
2. Subscribe to the stream that will receive RSS updates (default stream: rss)
3. create a ~/.zuliprc as described on https://zulipchat.com/api#api_keys
4. Test the script by running it manually, like this:
/usr/local/share/zulip/integrations/rss/rss-bot
You can customize the location on the feed file and recipient stream, e.g.:
/usr/local/share/zulip/integrations/rss/rss-bot --feed-file=/path/to/my-feeds --stream=my-rss-stream
4. Configure a crontab entry for this script. A sample crontab entry for
processing feeds stored in the default location and sending to the default
stream every 5 minutes is:
*/5 * * * * /usr/local/share/zulip/integrations/rss/rss-bot"""
parser = optparse.OptionParser(usage) # type: optparse.OptionParser
parser.add_option('--stream',
dest='stream',
help='The stream to which to send RSS messages.',
default="rss",
action='store')
parser.add_option('--data-dir',
dest='data_dir',
help='The directory where feed metadata is stored',
default=os.path.join(RSS_DATA_DIR),
action='store')
parser.add_option('--feed-file',
dest='feed_file',
help='The file containing a list of RSS feed URLs to follow, one URL per line',
default=os.path.join(RSS_DATA_DIR, "rss-feeds"),
action='store')
parser.add_option('--unwrap',
dest='unwrap',
action='store_true',
help='Convert word-wrapped paragraphs into single lines',
default=False)
parser.add_option('--math',
dest='math',
action='store_true',
help='Convert $ to $$ (for KaTeX processing)',
default=False)
parser.add_option_group(zulip.generate_option_group(parser))
(opts, args) = parser.parse_args() # type: Tuple[Any, List[str]]
def mkdir_p(path):
# type: (str) -> None
# Python doesn't have an analog to `mkdir -p` < Python 3.2.
try:
os.makedirs(path)
except OSError as e:
if e.errno == errno.EEXIST and os.path.isdir(path):
pass
else:
raise
try:
mkdir_p(opts.data_dir)
except OSError:
# We can't write to the logfile, so just print and give up.
print("Unable to store RSS data at %s." % (opts.data_dir,), file=sys.stderr)
exit(1)
log_file = os.path.join(opts.data_dir, "rss-bot.log") # type: str
log_format = "%(asctime)s: %(message)s" # type: str
logging.basicConfig(format=log_format)
formatter = logging.Formatter(log_format) # type: logging.Formatter
file_handler = logging.FileHandler(log_file) # type: logging.FileHandler
file_handler.setFormatter(formatter)
logger = logging.getLogger(__name__) # type: logging.Logger
logger.setLevel(logging.DEBUG)
logger.addHandler(file_handler)
def log_error_and_exit(error):
# type: (str) -> None
logger.error(error)
logger.error(usage)
exit(1)
class MLStripper(HTMLParser):
def __init__(self):
# type: () -> None
self.reset()
self.fed = [] # type: List[str]
def handle_data(self, data):
# type: (str) -> None
self.fed.append(data)
def get_data(self):
# type: () -> str
return ''.join(self.fed)
def strip_tags(html):
# type: (str) -> str
stripper = MLStripper()
stripper.feed(html)
return stripper.get_data()
def compute_entry_hash(entry):
# type: (Dict[str, Any]) -> str
entry_time = entry.get("published", entry.get("updated"))
entry_id = entry.get("id", entry.get("link"))
return hashlib.md5(entry_id + str(entry_time)).hexdigest()
def unwrap_text(body):
# type: (str) -> str
# Replace \n by space if it is preceded and followed by a non-\n.
# Example: '\na\nb\nc\n\nd\n' -> '\na b c\n\nd\n'
return re.sub('(?<=[^\n])\n(?=[^\n])', ' ', body)
def elide_subject(subject):
# type: (str) -> str
MAX_TOPIC_LENGTH = 60
if len(subject) > MAX_TOPIC_LENGTH:
subject = subject[:MAX_TOPIC_LENGTH - 3].rstrip() + '...'
return subject
def send_zulip(entry, feed_name):
# type: (Any, str) -> Dict[str, Any]
body = entry.summary # type: str
if opts.unwrap:
body = unwrap_text(body)
content = "**[%s](%s)**\n%s\n%s" % (entry.title,
entry.link,
strip_tags(body),
entry.link) # type: str
if opts.math:
content = content.replace('$', '$$')
message = {"type": "stream",
"sender": opts.zulip_email,
"to": opts.stream,
"subject": elide_subject(feed_name),
"content": content,
} # type: Dict[str, str]
return client.send_message(message)
try:
with open(opts.feed_file, "r") as f:
feed_urls = [feed.strip() for feed in f.readlines()] # type: List[str]
except IOError:
log_error_and_exit("Unable to read feed file at %s." % (opts.feed_file,))
client = zulip.Client(email=opts.zulip_email, api_key=opts.zulip_api_key,
site=opts.zulip_site, client="ZulipRSS/" + VERSION) # type: zulip.Client
first_message = True # type: bool
for feed_url in feed_urls:
feed_file = os.path.join(opts.data_dir, urllib.parse.urlparse(feed_url).netloc) # Type: str
try:
with open(feed_file, "r") as f:
old_feed_hashes = dict((line.strip(), True) for line in f.readlines()) # type: Dict[str, bool]
except IOError:
old_feed_hashes = {}
new_hashes = [] # type: List[str]
data = feedparser.parse(feed_url) # type: feedparser.parse
for entry in data.entries:
entry_hash = compute_entry_hash(entry) # type: str
# An entry has either been published or updated.
entry_time = entry.get("published_parsed", entry.get("updated_parsed")) # type: Tuple[int, int]
if entry_time is not None and (time.time() - calendar.timegm(entry_time)) > OLDNESS_THRESHOLD * 60 * 60 * 24:
# As a safeguard against misbehaving feeds, don't try to process
# entries older than some threshold.
continue
if entry_hash in old_feed_hashes:
# We've already seen this. No need to process any older entries.
break
if (not old_feed_hashes) and (len(new_hashes) >= 3):
# On a first run, pick up the 3 most recent entries. An RSS feed has
# entries in reverse chronological order.
break
feed_name = data.feed.title or feed_url # type: str
response = send_zulip(entry, feed_name) # type: Dict[str, Any]
if response["result"] != "success":
logger.error("Error processing %s" % (feed_url,))
logger.error(str(response))
if first_message:
# This is probably some fundamental problem like the stream not
# existing or something being misconfigured, so bail instead of
# getting the same error for every RSS entry.
log_error_and_exit("Failed to process first message")
# Go ahead and move on -- perhaps this entry is corrupt.
new_hashes.append(entry_hash)
first_message = False
with open(feed_file, "a") as f:
for hash in new_hashes:
f.write(hash + "\n")
logger.info("Sent zulips for %d %s entries" % (len(new_hashes), feed_url))

127
zulip/integrations/slack/zulip_slack.py Executable file
View file

@ -0,0 +1,127 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
#
# slacker is a dependency for this script.
#
from __future__ import absolute_import
from __future__ import print_function
import sys
import string
import random
from six.moves import range
from typing import List, Dict
import zulip
from slacker import Slacker, Response, Error as SlackError
import zulip_slack_config as config
client = zulip.Client(email=config.ZULIP_USER, api_key=config.ZULIP_API_KEY, site=config.ZULIP_SITE)
class FromSlackImporter(object):
def __init__(self, slack_token, get_archived_channels=True):
# type: (str, bool) -> None
self.slack = Slacker(slack_token)
self.get_archived_channels = get_archived_channels
self._check_slack_token()
def get_slack_users_email(self):
# type: () -> Dict[str, Dict[str, str]]
r = self.slack.users.list()
self._check_if_response_is_successful(r)
results_dict = {}
for user in r.body['members']:
if user['profile'].get('email') and user.get('deleted') is False:
results_dict[user['id']] = {'email': user['profile']['email'], 'name': user['profile']['real_name']}
return results_dict
def get_slack_public_channels_names(self):
# type: () -> List[Dict[str, str]]
r = self.slack.channels.list()
self._check_if_response_is_successful(r)
return [{'name': channel['name'], 'members': channel['members']} for channel in r.body['channels']]
def get_slack_private_channels_names(self):
# type: () -> List[str]
r = self.slack.groups.list()
self._check_if_response_is_successful(r)
return [
channel['name'] for channel in r.body['groups']
if not channel['is_archived'] or self.get_archived_channels
]
def _check_slack_token(self):
# type: () -> None
try:
r = self.slack.api.test()
self._check_if_response_is_successful(r)
except SlackError as e:
print(e)
sys.exit(1)
except Exception as e:
print(e)
sys.exit(1)
def _check_if_response_is_successful(self, response):
# type: (Response) -> None
print(response)
if not response.successful:
print(response.error)
sys.exit(1)
def _generate_random_password(size=10):
# type: (int) -> str
return ''.join(random.choice(string.ascii_letters + string.digits) for _ in range(size))
def get_and_add_users(slack_importer):
# type: (Slacker) -> Dict[str, Dict[str, str]]
users = slack_importer.get_slack_users_email()
added_users = {}
print('######### IMPORTING USERS STARTED #########\n')
for user_id, user in users.items():
r = client.create_user({
'email': user['email'],
'full_name': user['name'],
'short_name': user['name']
})
if not r.get('msg'):
added_users[user_id] = user
print(u"{} -> {}\nCreated\n".format(user['name'], user['email']))
else:
print(u"{} -> {}\n{}\n".format(user['name'], user['email'], r.get('msg')))
print('######### IMPORTING USERS FINISHED #########\n')
return added_users
def create_streams_and_add_subscribers(slack_importer, added_users):
# type: (Slacker, Dict[str, Dict[str, str]]) -> None
channels_list = slack_importer.get_slack_public_channels_names()
print('######### IMPORTING STREAMS STARTED #########\n')
for stream in channels_list:
subscribed_users = [added_users[member]['email'] for member in stream['members'] if member in added_users.keys()]
if subscribed_users:
r = client.add_subscriptions([{"name": stream['name']}], principals=subscribed_users)
if not r.get('msg'):
print(u"{} -> created\n".format(stream['name']))
else:
print(u"{} -> {}\n".format(stream['name'], r.get('msg')))
else:
print(u"{} -> wasn't created\nNo subscribers\n".format(stream['name']))
print('######### IMPORTING STREAMS FINISHED #########\n')
def main():
# type: () -> None
importer = FromSlackImporter(config.SLACK_TOKEN)
added_users = get_and_add_users(importer)
create_streams_and_add_subscribers(importer, added_users)
if __name__ == '__main__':
main()

34
zulip/integrations/slack/zulip_slack_config.py Normal file
View file

@ -0,0 +1,34 @@
# -*- coding: utf-8 -*-
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
### REQUIRED CONFIGURATION ###
# Change these values to your Slack credentials.
SLACK_TOKEN = 'slack_token'
# Change these values to the credentials for your Slack bot.
ZULIP_USER = 'user-email@zulip.com'
ZULIP_API_KEY = 'user-email_api_key'
# Set this to your Zulip API server URI
ZULIP_SITE = 'https://zulip.example.com'

75
zulip/integrations/svn/post-commit Executable file
View file

@ -0,0 +1,75 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Zulip notification post-commit hook.
# Copyright © 2012-2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
#
# The "post-commit" script is run after a transaction is completed and a new
# revision is created. It is passed arguments on the command line in this
# form:
# <path> <revision>
# For example:
# /srv/svn/carols 1843
import os
import sys
import os.path
import pysvn
if False:
from typing import Any, Dict, List, Optional, Text, Tuple, Union
sys.path.insert(0, os.path.dirname(__file__))
import zulip_svn_config as config
VERSION = "0.9"
if config.ZULIP_API_PATH is not None:
sys.path.append(config.ZULIP_API_PATH)
import zulip
client = zulip.Client(
email=config.ZULIP_USER,
site=config.ZULIP_SITE,
api_key=config.ZULIP_API_KEY,
client="ZulipSVN/" + VERSION) # type: zulip.Client
svn = pysvn.Client() # type: pysvn.Client
path, rev = sys.argv[1:] # type: Tuple[Text, Text]
# since its a local path, prepend "file://"
path = "file://" + path
entry = svn.log(path, revision_end=pysvn.Revision(pysvn.opt_revision_kind.number, rev))[0] # type: Dict[Text, Any]
message = "**{0}** committed revision r{1} to `{2}`.\n\n> {3}".format(
entry['author'],
rev,
path.split('/')[-1],
entry['revprops']['svn:log']) # type: Text
destination = config.commit_notice_destination(path, rev) # type: Optional[Dict[Text, Text]]
message_data = {
"type": "stream",
"to": destination["stream"],
"subject": destination["subject"],
"content": message,
} # type: Dict[str, Any]
client.send_message(message_data)

56
zulip/integrations/svn/zulip_svn_config.py Normal file
View file

@ -0,0 +1,56 @@
# -*- coding: utf-8 -*-
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# Change these values to configure authentication for the plugin
ZULIP_USER = "svn-bot@example.com"
ZULIP_API_KEY = "0123456789abcdef0123456789abcdef"
# commit_notice_destination() lets you customize where commit notices
# are sent to with the full power of a Python function.
#
# It takes the following arguments:
# * path = the path to the svn repository on the server
# * commit = the commit id
#
# Returns a dictionary encoding the stream and subject to send the
# notification to (or None to send no notification).
#
# The default code below will send every commit except for the "evil-master-plan"
# and "my-super-secret-repository" repos to
# * stream "commits"
# * topic "branch_name"
def commit_notice_destination(path, commit):
repo = path.split('/')[-1]
if repo not in ["evil-master-plan", "my-super-secret-repository"]:
return dict(stream = "commits",
subject = u"%s" % (repo,))
# Return None for cases where you don't want a notice sent
return None
## If properly installed, the Zulip API should be in your import
## path, but if not, set a custom path below
ZULIP_API_PATH = None
# Set this to your Zulip server's API URI
ZULIP_SITE = "https://zulip.example.com"

137
zulip/integrations/trac/zulip_trac.py Normal file
View file

@ -0,0 +1,137 @@
# -*- coding: utf-8 -*-
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# Zulip trac plugin -- sends zulips when tickets change.
#
# Install by copying this file and zulip_trac_config.py to the trac
# plugins/ subdirectory, customizing the constants in
# zulip_trac_config.py, and then adding "zulip_trac" to the
# components section of the conf/trac.ini file, like so:
#
# [components]
# zulip_trac = enabled
#
# You may then need to restart trac (or restart Apache) for the bot
# (or changes to the bot) to actually be loaded by trac.
from trac.core import Component, implements
from trac.ticket import ITicketChangeListener
import sys
import os.path
sys.path.insert(0, os.path.dirname(__file__))
import zulip_trac_config as config
VERSION = "0.9"
if False:
from typing import Any, Dict
if config.ZULIP_API_PATH is not None:
sys.path.append(config.ZULIP_API_PATH)
import zulip
client = zulip.Client(
email=config.ZULIP_USER,
site=config.ZULIP_SITE,
api_key=config.ZULIP_API_KEY,
client="ZulipTrac/" + VERSION)
def markdown_ticket_url(ticket, heading="ticket"):
# type: (Any, str) -> str
return "[%s #%s](%s/%s)" % (heading, ticket.id, config.TRAC_BASE_TICKET_URL, ticket.id)
def markdown_block(desc):
# type: (str) -> str
return "\n\n>" + "\n> ".join(desc.split("\n")) + "\n"
def truncate(string, length):
# type: (str, int) -> str
if len(string) <= length:
return string
return string[:length - 3] + "..."
def trac_subject(ticket):
# type: (Any) -> str
return truncate("#%s: %s" % (ticket.id, ticket.values.get("summary")), 60)
def send_update(ticket, content):
# type: (Any, str) -> None
client.send_message({
"type": "stream",
"to": config.STREAM_FOR_NOTIFICATIONS,
"content": content,
"subject": trac_subject(ticket)
})
class ZulipPlugin(Component):
implements(ITicketChangeListener)
def ticket_created(self, ticket):
# type: (Any) -> None
"""Called when a ticket is created."""
content = "%s created %s in component **%s**, priority **%s**:\n" % \
(ticket.values.get("reporter"), markdown_ticket_url(ticket),
ticket.values.get("component"), ticket.values.get("priority"))
# Include the full subject if it will be truncated
if len(ticket.values.get("summary")) > 60:
content += "**%s**\n" % (ticket.values.get("summary"),)
if ticket.values.get("description") != "":
content += "%s" % (markdown_block(ticket.values.get("description")),)
send_update(ticket, content)
def ticket_changed(self, ticket, comment, author, old_values):
# type: (Any, str, str, Dict[str, Any]) -> None
"""Called when a ticket is modified.
`old_values` is a dictionary containing the previous values of the
fields that have changed.
"""
if not (set(old_values.keys()).intersection(set(config.TRAC_NOTIFY_FIELDS)) or
(comment and "comment" in set(config.TRAC_NOTIFY_FIELDS))):
return
content = "%s updated %s" % (author, markdown_ticket_url(ticket))
if comment:
content += ' with comment: %s\n\n' % (markdown_block(comment),)
else:
content += ":\n\n"
field_changes = []
for key in old_values.keys():
if key == "description":
content += '- Changed %s from %s\n\nto %s' % (key, markdown_block(old_values.get(key)),
markdown_block(ticket.values.get(key)))
elif old_values.get(key) == "":
field_changes.append('%s: => **%s**' % (key, ticket.values.get(key)))
elif ticket.values.get(key) == "":
field_changes.append('%s: **%s** => ""' % (key, old_values.get(key)))
else:
field_changes.append('%s: **%s** => **%s**' % (key, old_values.get(key),
ticket.values.get(key)))
content += ", ".join(field_changes)
send_update(ticket, content)
def ticket_deleted(self, ticket):
# type: (Any) -> None
"""Called when a ticket is deleted."""
content = "%s was deleted." % markdown_ticket_url(ticket, heading="Ticket")
send_update(ticket, content)

51
zulip/integrations/trac/zulip_trac_config.py Normal file
View file

@ -0,0 +1,51 @@
# -*- coding: utf-8 -*-
#
# Copyright © 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# See zulip_trac.py for installation and configuration instructions
# Change these constants to configure the plugin:
ZULIP_USER = "trac-bot@example.com"
ZULIP_API_KEY = "0123456789abcdef0123456789abcdef"
STREAM_FOR_NOTIFICATIONS = "trac"
TRAC_BASE_TICKET_URL = "https://trac.example.com/ticket"
# Most people find that having every change in Trac result in a
# notification is too noisy -- in particular, when someone goes
# through recategorizing a bunch of tickets, that can often be noisy
# and annoying. We solve this issue by only sending a notification
# for changes to the fields listed below.
#
# TRAC_NOTIFY_FIELDS lets you specify which fields will trigger a
# Zulip notification in response to a trac update; you should change
# this list to match your team's workflow. The complete list of
# possible fields is:
#
# (priority, milestone, cc, owner, keywords, component, severity,
# type, versions, description, resolution, summary, comment)
TRAC_NOTIFY_FIELDS = ["description", "summary", "resolution", "comment", "owner"]
## If properly installed, the Zulip API should be in your import
## path, but if not, set a custom path below
ZULIP_API_PATH = None
# Set this to your Zulip API server URI
ZULIP_SITE = "https://zulip.example.com"

168
zulip/integrations/twitter/twitter-bot Executable file
View file

@ -0,0 +1,168 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Twitter integration for Zulip
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import os
import sys
import optparse
from six.moves.configparser import ConfigParser, NoSectionError, NoOptionError
import zulip
VERSION = "0.9"
CONFIGFILE = os.path.expanduser("~/.zulip_twitterrc")
def write_config(config, since_id, user):
# type: (ConfigParser, int, int) -> None
config.set('twitter', 'since_id', since_id)
config.set('twitter', 'user_id', user)
with open(CONFIGFILE, 'wb') as configfile:
config.write(configfile)
parser = optparse.OptionParser(r"""
%prog --user foo@example.com --api-key 0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 --twitter-id twitter_handle --site=https://zulip.example.com
Slurp tweets on your timeline into a specific zulip stream.
Run this on your personal machine. Your API key and twitter id
are revealed to local users through the command line or config
file.
This bot uses OAuth to authenticate with twitter. Please create a
~/.zulip_twitterrc with the following contents:
[twitter]
consumer_key =
consumer_secret =
access_token_key =
access_token_secret =
In order to obtain a consumer key & secret, you must register a
new application under your twitter account:
1. Go to http://dev.twitter.com
2. Log in
3. In the menu under your username, click My Applications
4. Create a new application
Make sure to go the application you created and click "create my
access token" as well. Fill in the values displayed.
Depends on: https://github.com/bear/python-twitter version 3.1
(`pip install python-twitter`)
""")
parser.add_option('--twitter-id',
help='Twitter username to poll for new tweets from"',
metavar='URL')
parser.add_option('--stream',
help='Default zulip stream to write tweets to')
parser.add_option('--limit-tweets',
default=15,
type='int',
help='Maximum number of tweets to push at once')
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
if not options.twitter_id:
parser.error('You must specify --twitter-id')
try:
config = ConfigParser()
config.read(CONFIGFILE)
consumer_key = config.get('twitter', 'consumer_key')
consumer_secret = config.get('twitter', 'consumer_secret')
access_token_key = config.get('twitter', 'access_token_key')
access_token_secret = config.get('twitter', 'access_token_secret')
except (NoSectionError, NoOptionError):
parser.error("Please provide a ~/.zulip_twitterrc")
if not consumer_key or not consumer_secret or not access_token_key or not access_token_secret:
parser.error("Please provide a ~/.zulip_twitterrc")
try:
import twitter
except ImportError:
parser.error("Please install twitter-python")
api = twitter.Api(consumer_key=consumer_key,
consumer_secret=consumer_secret,
access_token_key=access_token_key,
access_token_secret=access_token_secret)
user = api.VerifyCredentials()
if not user.id:
print("Unable to log in to twitter with supplied credentials. Please double-check and try again")
sys.exit()
try:
since_id = config.getint('twitter', 'since_id')
except NoOptionError:
since_id = -1
try:
user_id = config.get('twitter', 'user_id')
except NoOptionError:
user_id = options.twitter_id
client = zulip.Client(
email=options.zulip_email,
api_key=options.zulip_api_key,
site=options.zulip_site,
client="ZulipTwitter/" + VERSION,
verbose=True)
if since_id < 0 or options.twitter_id != user_id:
# No since id yet, fetch the latest and then start monitoring from next time
# Or, a different user id is being asked for, so start from scratch
# Either way, fetch last 5 tweets to start off
statuses = api.GetUserTimeline(screen_name=options.twitter_id, count=5)
else:
# We have a saved last id, so insert all newer tweets into the zulip stream
statuses = api.GetUserTimeline(screen_name=options.twitter_id, since_id=since_id)
for status in statuses[::-1][:options.limit_tweets]:
composed = "%s (%s)" % (status.user.name, status.user.screen_name)
message = {
"type": "stream",
"to": [options.stream],
"subject": composed,
"content": status.text,
}
ret = client.send_message(message)
if ret['result'] == 'error':
# If sending failed (e.g. no such stream), abort and retry next time
print("Error sending message to zulip: %s" % ret['msg'])
break
else:
since_id = status.id
write_config(config, since_id, user_id)

194
zulip/integrations/twitter/twitter-search-bot Executable file
View file

@ -0,0 +1,194 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# Twitter search integration for Zulip
#
# Copyright © 2014 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
from __future__ import print_function
import os
import sys
import optparse
from six.moves.configparser import ConfigParser, NoSectionError, NoOptionError
import zulip
VERSION = "0.9"
CONFIGFILE = os.path.expanduser("~/.zulip_twitterrc")
def write_config(config, since_id):
# type: (ConfigParser, int) -> None
if 'search' not in config.sections():
config.add_section('search')
config.set('search', 'since_id', since_id)
with open(CONFIGFILE, 'wb') as configfile:
config.write(configfile)
parser = optparse.OptionParser(r"""
%prog --user username@example.com --api-key a0b1c2d3e4f5a6b7c8d9e0f1a2b3c4d5 \
--search="@nprnews,quantum physics"
Send Twitter search results to a Zulip stream.
Depends on: https://github.com/bear/python-twitter version 3.1
To use this script:
0. Use `pip install python-twitter` to install `python-twitter`.
1. Set up Twitter authentication, as described below
2. Subscribe to the stream that will receive Twitter updates (default stream: twitter)
3. Test the script by running it manually, like this:
/usr/local/share/zulip/integrations/twitter/twitter-search-bot \
--search="@nprnews,quantum physics" --site=https://zulip.example.com
4. Configure a crontab entry for this script. A sample crontab entry
that will process tweets every 5 minutes is:
*/5 * * * * /usr/local/share/zulip/integrations/twitter/twitter-search-bot [options]
== Setting up Twitter authentications ==
Run this on a personal or trusted machine, because your API key is
visible to local users through the command line or config file.
This bot uses OAuth to authenticate with Twitter. Please create a
~/.zulip_twitterrc with the following contents:
[twitter]
consumer_key =
consumer_secret =
access_token_key =
access_token_secret =
In order to obtain a consumer key & secret, you must register a
new application under your Twitter account:
1. Go to http://dev.twitter.com
2. Log in
3. In the menu under your username, click My Applications
4. Create a new application
Make sure to go the application you created and click "create my
access token" as well. Fill in the values displayed.
""")
parser.add_option('--search',
dest='search_terms',
help='Terms to search on',
action='store')
parser.add_option('--stream',
dest='stream',
help='The stream to which to send tweets',
default="twitter",
action='store')
parser.add_option('--limit-tweets',
default=15,
type='int',
help='Maximum number of tweets to send at once')
parser.add_option_group(zulip.generate_option_group(parser))
(opts, args) = parser.parse_args()
if not opts.search_terms:
parser.error('You must specify a search term.')
try:
config = ConfigParser()
config.read(CONFIGFILE)
consumer_key = config.get('twitter', 'consumer_key')
consumer_secret = config.get('twitter', 'consumer_secret')
access_token_key = config.get('twitter', 'access_token_key')
access_token_secret = config.get('twitter', 'access_token_secret')
except (NoSectionError, NoOptionError):
parser.error("Please provide a ~/.zulip_twitterrc")
if not (consumer_key and consumer_secret and access_token_key and access_token_secret):
parser.error("Please provide a ~/.zulip_twitterrc")
try:
since_id = config.getint('search', 'since_id')
except (NoOptionError, NoSectionError):
since_id = 0
try:
import twitter
except ImportError:
parser.error("Please install twitter-python")
api = twitter.Api(consumer_key=consumer_key,
consumer_secret=consumer_secret,
access_token_key=access_token_key,
access_token_secret=access_token_secret)
user = api.VerifyCredentials()
if not user.id:
print("Unable to log in to twitter with supplied credentials. Please double-check and try again")
sys.exit()
client = zulip.Client(
email=opts.zulip_email,
api_key=opts.zulip_api_key,
site=opts.zulip_site,
client="ZulipTwitterSearch/" + VERSION,
verbose=True)
search_query = " OR ".join(opts.search_terms.split(","))
statuses = api.GetSearch(search_query, since_id=since_id)
for status in statuses[::-1][:opts.limit_tweets]:
# https://twitter.com/eatevilpenguins/status/309995853408530432
composed = "%s (%s)" % (status.user.name,
status.user.screen_name)
url = "https://twitter.com/%s/status/%s" % (status.user.screen_name,
status.id)
content = status.text
search_term_used = None
for term in opts.search_terms.split(","):
if term.lower() in content.lower():
search_term_used = term
break
# For some reason (perhaps encodings or message tranformations we
# didn't anticipate), we don't know what term was used, so use a
# default.
if not search_term_used:
search_term_used = "mentions"
message = {
"type": "stream",
"to": [opts.stream],
"subject": search_term_used,
"content": url,
}
ret = client.send_message(message)
if ret['result'] == 'error':
# If sending failed (e.g. no such stream), abort and retry next time
print("Error sending message to zulip: %s" % ret['msg'])
break
else:
since_id = status.id
write_config(config, since_id)

366
zulip/integrations/zephyr/check-mirroring Executable file
View file

@ -0,0 +1,366 @@
#!/usr/bin/env python
from __future__ import print_function
from __future__ import absolute_import
import sys
import time
import optparse
import os
import random
import logging
import subprocess
import hashlib
from six.moves import range
if False:
from typing import Any, Dict, List, Set, Tuple
parser = optparse.OptionParser()
parser.add_option('--verbose',
dest='verbose',
default=False,
action='store_true')
parser.add_option('--site',
dest='site',
default=None,
action='store')
parser.add_option('--sharded',
default=False,
action='store_true')
parser.add_option('--root-path',
dest='root_path',
default="/home/zulip",
action='store')
(options, args) = parser.parse_args()
# The 'api' directory needs to go first, so that 'import zulip' won't pick up
# some other directory named 'zulip'.
pyzephyr_lib_path = "python-zephyr/build/lib.linux-%s-%s/" % (os.uname()[4], sys.version[0:3])
sys.path[:0] = [os.path.join(options.root_path, "api/"),
os.path.join(options.root_path, "python-zephyr"),
os.path.join(options.root_path, pyzephyr_lib_path),
options.root_path]
mit_user = 'tabbott/extra@ATHENA.MIT.EDU'
sys.path.append(".")
import zulip
zulip_client = zulip.Client(
verbose=True,
client="ZulipMonitoring/0.1",
site=options.site)
# Configure logging
log_file = "/var/log/zulip/check-mirroring-log"
log_format = "%(asctime)s: %(message)s"
logging.basicConfig(format=log_format)
formatter = logging.Formatter(log_format)
file_handler = logging.FileHandler(log_file)
file_handler.setFormatter(formatter)
logger = logging.getLogger(__name__)
logger.setLevel(logging.DEBUG)
logger.addHandler(file_handler)
# Initialize list of streams to test
if options.sharded:
# NOTE: Streams in this list must be in the zulip_user's Zulip
# subscriptions, or we won't receive messages via Zulip.
# The sharded stream list has a bunch of pairs
# (stream, shard_name), where sha1sum(stream).startswith(shard_name)
test_streams = [
("message", "p"),
("tabbott-nagios-test-32", "0"),
("tabbott-nagios-test-33", "1"),
("tabbott-nagios-test-2", "2"),
("tabbott-nagios-test-5", "3"),
("tabbott-nagios-test-13", "4"),
("tabbott-nagios-test-7", "5"),
("tabbott-nagios-test-22", "6"),
("tabbott-nagios-test-35", "7"),
("tabbott-nagios-test-4", "8"),
("tabbott-nagios-test-3", "9"),
("tabbott-nagios-test-1", "a"),
("tabbott-nagios-test-49", "b"),
("tabbott-nagios-test-34", "c"),
("tabbott-nagios-test-12", "d"),
("tabbott-nagios-test-11", "e"),
("tabbott-nagios-test-9", "f"),
]
for (stream, test) in test_streams:
if stream == "message":
continue
assert(hashlib.sha1(stream.encode("utf-8")).hexdigest().startswith(test))
else:
test_streams = [
("message", "p"),
("tabbott-nagios-test", "a"),
]
def print_status_and_exit(status):
# type: (int) -> None
# The output of this script is used by Nagios. Various outputs,
# e.g. true success and punting due to a SERVNAK, result in a
# non-alert case, so to give us something unambiguous to check in
# Nagios, print the exit status.
print(status)
sys.exit(status)
def send_zulip(message):
# type: (Dict[str, str]) -> None
result = zulip_client.send_message(message)
if result["result"] != "success":
logger.error("Error sending zulip, args were:")
logger.error(str(message))
logger.error(str(result))
print_status_and_exit(1)
# Returns True if and only if we "Detected server failure" sending the zephyr.
def send_zephyr(zwrite_args, content):
# type: (List[str], str) -> bool
p = subprocess.Popen(zwrite_args, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate(input=content.encode("utf-8"))
if p.returncode != 0:
if "Detected server failure while receiving acknowledgement for" in stdout:
logger.warning("Got server failure error sending zephyr; retrying")
logger.warning(stderr)
return True
logger.error("Error sending zephyr:")
logger.info(stdout)
logger.error(stderr)
print_status_and_exit(1)
return False
# Subscribe to Zulip
try:
res = zulip_client.register(event_types=["message"])
if 'error' in res.get('result'):
logging.error("Error subscribing to Zulips!")
logging.error(res['msg'])
print_status_and_exit(1)
queue_id, last_event_id = (res['queue_id'], res['last_event_id'])
except Exception:
logger.exception("Unexpected error subscribing to Zulips")
print_status_and_exit(1)
# Subscribe to Zephyrs
import zephyr
zephyr_subs_to_add = []
for (stream, test) in test_streams:
if stream == "message":
zephyr_subs_to_add.append((stream, 'personal', mit_user))
else:
zephyr_subs_to_add.append((stream, '*', '*'))
actually_subscribed = False
for tries in range(10):
try:
zephyr.init()
zephyr._z.subAll(zephyr_subs_to_add)
zephyr_subs = zephyr._z.getSubscriptions()
missing = 0
for elt in zephyr_subs_to_add:
if elt not in zephyr_subs:
logging.error("Failed to subscribe to %s" % (elt,))
missing += 1
if missing == 0:
actually_subscribed = True
break
except IOError as e:
if "SERVNAK received" in e: # type: ignore # https://github.com/python/mypy/issues/2118
logger.error("SERVNAK repeatedly received, punting rest of test")
else:
logger.exception("Exception subscribing to zephyrs")
if not actually_subscribed:
logger.error("Failed to subscribe to zephyrs")
print_status_and_exit(1)
# Prepare keys
zhkeys = {} # type: Dict[str, Tuple[str, str]]
hzkeys = {} # type: Dict[str, Tuple[str, str]]
def gen_key(key_dict):
# type: (Dict[str, Any]) -> str
bits = str(random.getrandbits(32))
while bits in key_dict:
# Avoid the unlikely event that we get the same bits twice
bits = str(random.getrandbits(32))
return bits
def gen_keys(key_dict):
# type: (Dict[str, Tuple[str, str]]) -> None
for (stream, test) in test_streams:
key_dict[gen_key(key_dict)] = (stream, test)
gen_keys(zhkeys)
gen_keys(hzkeys)
notices = []
# We check for new zephyrs multiple times, to avoid filling the zephyr
# receive queue with 30+ messages, which might result in messages
# being dropped.
def receive_zephyrs():
# type: () -> None
while True:
try:
notice = zephyr.receive(block=False)
except Exception:
logging.exception("Exception receiving zephyrs:")
notice = None
if notice is None:
break
if notice.opcode != "":
continue
notices.append(notice)
logger.info("Starting sending messages!")
# Send zephyrs
zsig = "Timothy Good Abbott"
for key, (stream, test) in zhkeys.items():
if stream == "message":
zwrite_args = ["zwrite", "-n", "-s", zsig, mit_user]
else:
zwrite_args = ["zwrite", "-n", "-s", zsig, "-c", stream, "-i", "test"]
server_failure = send_zephyr(zwrite_args, str(key))
if server_failure:
# Replace the key we're not sure was delivered with a new key
value = zhkeys.pop(key)
new_key = gen_key(zhkeys)
zhkeys[new_key] = value
server_failure_again = send_zephyr(zwrite_args, str(new_key))
if server_failure_again:
logging.error("Zephyr server failure twice in a row on keys %s and %s! Aborting." %
(key, new_key))
print_status_and_exit(1)
else:
logging.warning("Replaced key %s with %s due to Zephyr server failure." %
(key, new_key))
receive_zephyrs()
receive_zephyrs()
logger.info("Sent Zephyr messages!")
# Send Zulips
for key, (stream, test) in hzkeys.items():
if stream == "message":
send_zulip({
"type": "private",
"content": str(key),
"to": zulip_client.email,
})
else:
send_zulip({
"type": "stream",
"subject": "test",
"content": str(key),
"to": stream,
})
receive_zephyrs()
logger.info("Sent Zulip messages!")
# Normally messages manage to forward through in under 3 seconds, but
# sleep 10 to give a safe margin since the messages do need to do 2
# round trips. This alert is for correctness, not performance, and so
# we want it to reliably alert only when messages aren't being
# delivered at all.
time.sleep(10)
receive_zephyrs()
logger.info("Starting receiving messages!")
# receive zulips
res = zulip_client.get_events(queue_id=queue_id, last_event_id=last_event_id)
if 'error' in res.get('result'):
logging.error("Error subscribing to Zulips!")
logging.error(res['msg'])
print_status_and_exit(1)
messages = [event['message'] for event in res['events']]
logger.info("Finished receiving Zulip messages!")
receive_zephyrs()
logger.info("Finished receiving Zephyr messages!")
all_keys = set(list(zhkeys.keys()) + list(hzkeys.keys()))
def process_keys(content_list):
# type: (List[str]) -> Tuple[Dict[str, int], Set[str], Set[str], bool, bool]
# Start by filtering out any keys that might have come from
# concurrent check-mirroring processes
content_keys = [key for key in content_list if key in all_keys]
key_counts = {} # type: Dict[str, int]
for key in all_keys:
key_counts[key] = 0
for key in content_keys:
key_counts[key] += 1
z_missing = set(key for key in zhkeys.keys() if key_counts[key] == 0)
h_missing = set(key for key in hzkeys.keys() if key_counts[key] == 0)
duplicates = any(val > 1 for val in key_counts.values())
success = all(val == 1 for val in key_counts.values())
return key_counts, z_missing, h_missing, duplicates, success
# The h_foo variables are about the messages we _received_ in Zulip
# The z_foo variables are about the messages we _received_ in Zephyr
h_contents = [message["content"] for message in messages]
z_contents = [notice.message.split('\0')[1] for notice in notices]
(h_key_counts, h_missing_z, h_missing_h, h_duplicates, h_success) = process_keys(h_contents)
(z_key_counts, z_missing_z, z_missing_h, z_duplicates, z_success) = process_keys(z_contents)
if z_success and h_success:
logger.info("Success!")
print_status_and_exit(0)
elif z_success:
logger.info("Received everything correctly in Zephyr!")
elif h_success:
logger.info("Received everything correctly in Zulip!")
logger.error("Messages received the wrong number of times:")
for key in all_keys:
if z_key_counts[key] == 1 and h_key_counts[key] == 1:
continue
if key in zhkeys:
(stream, test) = zhkeys[key]
logger.warning("%10s: z got %s, h got %s. Sent via Zephyr(%s): class %s" %
(key, z_key_counts[key], h_key_counts[key], test, stream))
if key in hzkeys:
(stream, test) = hzkeys[key]
logger.warning("%10s: z got %s. h got %s. Sent via Zulip(%s): class %s" %
(key, z_key_counts[key], h_key_counts[key], test, stream))
logger.error("")
logger.error("Summary of specific problems:")
if h_duplicates:
logger.error("zulip: Received duplicate messages!")
logger.error("zulip: This is probably a bug in our message loop detection.")
logger.error("zulip: where Zulips go zulip=>zephyr=>zulip")
if z_duplicates:
logger.error("zephyr: Received duplicate messages!")
logger.error("zephyr: This is probably a bug in our message loop detection.")
logger.error("zephyr: where Zephyrs go zephyr=>zulip=>zephyr")
if z_missing_z:
logger.error("zephyr: Didn't receive all the Zephyrs we sent on the Zephyr end!")
logger.error("zephyr: This is probably an issue with check-mirroring sending or receiving Zephyrs.")
if h_missing_h:
logger.error("zulip: Didn't receive all the Zulips we sent on the Zulip end!")
logger.error("zulip: This is probably an issue with check-mirroring sending or receiving Zulips.")
if z_missing_h:
logger.error("zephyr: Didn't receive all the Zulips we sent on the Zephyr end!")
if z_missing_h == h_missing_h:
logger.error("zephyr: Including some Zulips that we did receive on the Zulip end.")
logger.error("zephyr: This suggests we have a zulip=>zephyr mirroring problem.")
logger.error("zephyr: aka the personals mirroring script has issues.")
if h_missing_z:
logger.error("zulip: Didn't receive all the Zephyrs we sent on the Zulip end!")
if h_missing_z == z_missing_z:
logger.error("zulip: Including some Zephyrs that we did receive on the Zephyr end.")
logger.error("zulip: This suggests we have a zephyr=>zulip mirroring problem.")
logger.error("zulip: aka the global class mirroring script has issues.")
zulip_client.deregister(queue_id)
print_status_and_exit(1)

42
zulip/integrations/zephyr/process_ccache Executable file
View file

@ -0,0 +1,42 @@
#!/usr/bin/env python
import sys
import subprocess
import base64
short_user = sys.argv[1]
api_key = sys.argv[2]
ccache_data_encoded = sys.argv[3]
# Update the Kerberos ticket cache file
program_name = "zmirror-%s" % (short_user,)
with open("/home/zulip/ccache/%s" % (program_name,), "w") as f:
f.write(base64.b64decode(ccache_data_encoded))
# Setup API key
api_key_path = "/home/zulip/api-keys/%s" % (program_name,)
open(api_key_path, "w").write(api_key + "\n")
# Setup supervisord configuration
supervisor_path = "/etc/supervisor/conf.d/%s.conf" % (program_name,)
template = "/home/zulip/zulip/api/integrations/zephyr/zmirror_private.conf.template"
template_data = open(template).read()
session_path = "/home/zulip/zephyr_sessions/%s" % (program_name,)
# Preserve mail zephyrs forwarding setting across rewriting the config file
try:
if "--forward-mail-zephyrs" in open(supervisor_path, "r").read():
template_data = template_data.replace("--use-sessions", "--use-sessions --forward-mail-zephyrs")
except Exception:
pass
open(supervisor_path, "w").write(template_data.replace("USERNAME", short_user))
# Delete your session
subprocess.check_call(["rm", "-f", session_path])
# Update your supervisor config, which may restart your mirror
subprocess.check_call(["supervisorctl", "reread"])
subprocess.check_call(["supervisorctl", "update"])
# Restart your mirror, in case it wasn't restarted by the previous
# (Otherwise if the mirror lost subs, this would do nothing)
# TODO: check whether we JUST restarted it first
subprocess.check_call(["supervisorctl", "restart", program_name])

74
zulip/integrations/zephyr/sync-public-streams Executable file
View file

@ -0,0 +1,74 @@
#!/usr/bin/env python
import sys
import os
import logging
import optparse
import time
import simplejson
import subprocess
import unicodedata
sys.path.append(os.path.join(os.path.dirname(__file__), '..', 'api'))
import zulip
from typing import Set
def fetch_public_streams():
# type: () -> Set[bytes]
public_streams = set()
try:
res = zulip_client.get_streams(include_all_active=True)
if res.get("result") == "success":
streams = res["streams"]
else:
logging.error("Error getting public streams:\n%s" % (res,))
return None
except Exception:
logging.exception("Error getting public streams:")
return None
for stream in streams:
stream_name = stream["name"]
# Zephyr class names are canonicalized by first applying NFKC
# normalization and then lower-casing server-side
canonical_cls = unicodedata.normalize("NFKC", stream_name).lower().encode("utf-8")
if canonical_cls in [b'security', b'login', b'network', b'ops', b'user_locate',
b'mit', b'moof', b'wsmonitor', b'wg_ctl', b'winlogger',
b'hm_ctl', b'hm_stat', b'zephyr_admin', b'zephyr_ctl']:
# These zephyr classes cannot be subscribed to by us, due
# to MIT's Zephyr access control settings
continue
public_streams.add(canonical_cls)
return public_streams
if __name__ == "__main__":
log_file = "/home/zulip/sync_public_streams.log"
logger = logging.getLogger(__name__)
log_format = "%(asctime)s: %(message)s"
logging.basicConfig(format=log_format)
formatter = logging.Formatter(log_format)
logger.setLevel(logging.DEBUG)
file_handler = logging.FileHandler(log_file)
file_handler.setFormatter(formatter)
logger.addHandler(file_handler)
parser = optparse.OptionParser()
parser.add_option_group(zulip.generate_option_group(parser))
(options, args) = parser.parse_args()
zulip_client = zulip.Client(client="ZulipSyncPublicStreamsBot/0.1")
while True:
time.sleep(15)
public_streams = fetch_public_streams()
if public_streams is None:
continue
f = open("/home/zulip/public_streams.tmp", "w")
f.write(simplejson.dumps(list(public_streams)) + "\n")
f.close()
subprocess.call(["mv", "/home/zulip/public_streams.tmp", "/home/zulip/public_streams"])

93
zulip/integrations/zephyr/zephyr_mirror.py Executable file
View file

@ -0,0 +1,93 @@
#!/usr/bin/env python
# Copyright (C) 2012 Zulip, Inc.
#
# Permission is hereby granted, free of charge, to any person
# obtaining a copy of this software and associated documentation files
# (the "Software"), to deal in the Software without restriction,
# including without limitation the rights to use, copy, modify, merge,
# publish, distribute, sublicense, and/or sell copies of the Software,
# and to permit persons to whom the Software is furnished to do so,
# subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
# EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
# BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
# ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
# CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
from __future__ import absolute_import
from __future__ import print_function
import sys
import subprocess
import os
import traceback
import signal
sys.path[:0] = [os.path.dirname(__file__)]
from zephyr_mirror_backend import parse_args
(options, args) = parse_args()
sys.path[:0] = [os.path.join(options.root_path, 'api')]
from types import FrameType
from typing import Any
def die(signal, frame):
# type: (int, FrameType) -> None
# We actually want to exit, so run os._exit (so as not to be caught and restarted)
os._exit(1)
signal.signal(signal.SIGINT, die)
from zulip import RandomExponentialBackoff
args = [os.path.join(options.root_path, "user_root", "zephyr_mirror_backend.py")]
args.extend(sys.argv[1:])
if options.sync_subscriptions:
subprocess.call(args)
sys.exit(0)
if options.forward_class_messages and not options.noshard:
sys.path.append("/home/zulip/zulip")
if options.on_startup_command is not None:
subprocess.call([options.on_startup_command])
from zerver.lib.parallel import run_parallel
print("Starting parallel zephyr class mirroring bot")
jobs = list("0123456789abcdef")
def run_job(shard):
# type: (str) -> int
subprocess.call(args + ["--shard=%s" % (shard,)])
return 0
for (status, job) in run_parallel(run_job, jobs, threads=16):
print("A mirroring shard died!")
pass
sys.exit(0)
backoff = RandomExponentialBackoff(timeout_success_equivalent=300)
while backoff.keep_going():
print("Starting zephyr mirroring bot")
try:
subprocess.call(args)
except Exception:
traceback.print_exc()
backoff.fail()
error_message = """
ERROR: The Zephyr mirroring bot is unable to continue mirroring Zephyrs.
This is often caused by failing to maintain unexpired Kerberos tickets
or AFS tokens. See https://zulipchat.com/zephyr for documentation on how to
maintain unexpired Kerberos tickets and AFS tokens.
"""
print(error_message)
sys.exit(1)

1194
zulip/integrations/zephyr/zephyr_mirror_backend.py Executable file
View file

File diff suppressed because it is too large Load diff

3
zulip/integrations/zephyr/zmirror-renew-kerberos Executable file
View file

@ -0,0 +1,3 @@
#!/bin/sh
krb_user_id=1051
env KRB5CCNAME=/tmp/krb5cc_"$krb_user_id".tmp kinit -k -t /home/zulip/tabbott.extra.keytab tabbott/extra@ATHENA.MIT.EDU; mv /tmp/krb5cc_"$krb_user_id".tmp /tmp/krb5cc_"$krb_user_id"

11
zulip/integrations/zephyr/zmirror_private.conf.template Normal file
View file

@ -0,0 +1,11 @@
[program:zmirror-USERNAME]
command=/home/zulip/zulip/api/integrations/zephyr/zephyr_mirror_backend.py --root-path=/home/zulip/zulip --user=USERNAME --log-path=/home/zulip/logs/mirror-log-%(program_name)s --use-sessions --session-path=/home/zulip/zephyr_sessions/%(program_name)s --api-key-file=/home/zulip/api-keys/%(program_name)s --ignore-expired-tickets --nagios-path=/home/zulip/mirror_status/%(program_name)s --nagios-class=zulip-mirror-nagios --site=https://zephyr.zulipchat.com
priority=200 ; the relative start priority (default 999)
autostart=true ; start at supervisord start (default: true)
autorestart=true ; whether/when to restart (default: unexpected)
stopsignal=TERM ; signal used to kill process (default TERM)
stopwaitsecs=30 ; max num secs to wait b4 SIGKILL (default 10)
user=zulip ; setuid to this UNIX account to run the program
redirect_stderr=true ; redirect proc stderr to stdout (default false)
stdout_logfile=/var/log/zulip/%(program_name)s.log ; stdout log path, NONE for none; default AUTO
environment=HOME="/home/zulip",USER="zulip",KRB5CCNAME="/home/zulip/ccache/%(program_name)s"

3
zulip/setup.cfg Normal file
View file

@ -0,0 +1,3 @@
# This way our scripts are installed in the data directory
[aliases]
install = install_data install

98
zulip/setup.py Executable file
View file

@ -0,0 +1,98 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function
if False:
from typing import Any, Dict, Generator, List, Tuple
import os
import sys
import itertools
def version():
# type: () -> str
version_py = os.path.join(os.path.dirname(__file__), "zulip", "__init__.py")
with open(version_py) as in_handle:
version_line = next(itertools.dropwhile(lambda x: not x.startswith("__version__"),
in_handle))
version = version_line.split('=')[-1].strip().replace('"', '')
return version
def recur_expand(target_root, dir):
# type: (Any, Any) -> Generator[Tuple[str, List[str]], None, None]
for root, _, files in os.walk(dir):
paths = [os.path.join(root, f) for f in files]
if len(paths):
yield os.path.join(target_root, root), paths
# We should be installable with either setuptools or distutils.
package_info = dict(
name='zulip',
version=version(),
description='Bindings for the Zulip message API',
author='Zulip Open Source Project',
author_email='zulip-devel@googlegroups.com',
classifiers=[
'Development Status :: 4 - Beta',
'Environment :: Web Environment',
'Intended Audience :: Developers',
'License :: OSI Approved :: MIT License',
'Topic :: Communications :: Chat',
],
url='https://www.zulip.org/',
data_files=[('share/zulip/examples',
["examples/zuliprc",
"examples/create-user",
"examples/edit-message",
"examples/get-presence",
"examples/get-public-streams",
"examples/list-members",
"examples/list-subscriptions",
"examples/print-events",
"examples/print-messages",
"examples/recent-messages",
"examples/send-message",
"examples/subscribe",
"examples/unsubscribe",
])] + list(recur_expand('share/zulip', 'integrations/')),
entry_points={
'console_scripts': [
'zulip-send=zulip.send:main',
],
},
) # type: Dict[str, Any]
setuptools_info = dict(
install_requires=['requests>=0.12.1',
'simplejson',
'six',
'typing>=3.5.2.2',
],
)
try:
from setuptools import setup, find_packages
package_info.update(setuptools_info)
package_info['packages'] = find_packages()
except ImportError:
from distutils.core import setup
from distutils.version import LooseVersion
# Manual dependency check
try:
import simplejson
except ImportError:
print("simplejson is not installed", file=sys.stderr)
sys.exit(1)
try:
import requests
assert(LooseVersion(requests.__version__) >= LooseVersion('0.12.1'))
except (ImportError, AssertionError):
print("requests >=0.12.1 is not installed", file=sys.stderr)
sys.exit(1)
package_info['packages'] = ['zulip']
setup(**package_info)

0
zulip/__init__.py → zulip/zulip/__init__.py
View file

3
zulip/send.py → zulip/zulip/send.py
View file

@ -23,14 +23,11 @@
# THE SOFTWARE.
import sys
import os
import optparse
import logging
from typing import Any, Dict, List, Optional
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
import zulip
logging.basicConfig()