We were having problems where we were suspiciously processing notices
at a rate of 1 notice per 15s, which suggests that we the select was
timing out even though there were notices to be fetched immediately.
We fix this by clearing the queue each time our select loop ends.
(imported from commit 7e7bfbb2126d1f4170d65d1483a0b799dcab80b9)
This should enabled us, in the future, to enable support for mail
zephyrs without requiring users to re-auth.
(imported from commit 2665743212da567fe85742d30cca42b902e41a0d)
We were intending to just be directing the logger logs, not logging
every message for potentially resending.
(imported from commit 9dd53e3968880745a8c01244db5d2c2247dfe85c)
This shows up when you're not running a Zephyr mirroring bot and lets
you use Webathena to have us run it. Obviously needs more docs.
Current problems include:
* supervisorctl reload ends up recreating /var/run/supervisor.sock
with the wrong permissions, so it only works once in a row before
you need to chmod that.
* /etc/supervisor/conf.d needs to be humbug-writeable; this is a clear
local root vulnerability
* This uses SSH and thus is kinda slow.
(imported from commit 7029979615ffd50b10f126ce2cf9a85a5eefd7a2)
The davidben-patched-for-roost Zephyr branch (available at
https://github.com/davidben/zephyr/tree/roost) adds Zephyr support for
these options. We also patch python-zephyr to expose them. These
basically let you save your Zephyr tickets and port number to a file,
so that you can later restore them (even potentially after the machine
rebooted). Basically because Zephyr is UDP, the Zephyr server will
continue trying to deliver messages to a particular port number that
was registered for up to 20 minutes after getting an error; so we can
even have downtime and reboot and still get our packets so long as we
restore the sessions within 20 minutes.
(imported from commit 986cbb157ddfa57aa4b644cd826f8418e9876dc7)
Previously it only provided the list of all public streams; now it
allows one to specify any union of some of the following:
* all public streams
* all streams the user subscribed to
(the most relevant being the union of those two, which is what we want
for the "streams" page).
Or:
* all streams in realm (superuser only)
The manual task required is that when this is pushed to prod, we need
to also deploy the new sync-public-streams version to zmirror.
(imported from commit 27848b8bd136e2777f399b7d05b2fdcec35e4e21)
Our .crypt-table parsing code isn't quite correct, in that we don't
handle either the "zcrypt default" or "zcrypt by class/instance" pair
options (for sending messages in either direction) -- you have to be
zcrypting for an entire class. I think this makes sense given that on
the Zulip end we can only enforce anything on a stream level.
(imported from commit a7901b1dc025a04a23ee71ecdd499e3f150ba614)
When we deploy this, we should remove the relevant jobs from root's
crontab on our app servers.
(imported from commit 749be952d504f5a4d243cf59f6430acc689fc821)
For now we only support the AES encryption type since the DES one is
probably not used anymore.
(imported from commit 222606db9f704917e74159e7d07a110187a236e6)
Just before this is pushed to prod, we need to rename the Humbug feedback
bot in the database using:
./manage.py change_user_email feedback@humbughq.comfeedback@zulip.com
/etc/init.d/memcached restart
and we also need to update and restart feedback-bot in its deployed
location.
No action is required on pushing this to staging, but in between when
this is pushed to staging and when it is pushed to prod (and that
transition performed), feedback will not work on staging.
(imported from commit 73fc36f680b978f3aebae5df1822918ae4d4e952)
When we push this to staging, we'll need to rename the bot in the
database and also pull on git.zulip.net.
(imported from commit 22b2397b197c8820f0e55daecd8f98d829e195bd)
This must be deployed after we update our running nginx configuration
to serve api.humbughq.com.
(imported from commit b5c34ebdd595f55eecd6dca6a18a37f105107bd5)
Since in the future we might want requests to add subscriptions to
include things like colors, in_home_view, etc., we're changing the
data format for the add_subscriptions API call to pass each stream as
a dictionary, giving a convenient place to put any added options.
The manual step required here is updating the API version in AFS
available for use with the zephyr_mirror.py system.
(imported from commit 364960cca582a0658f0d334668822045c001b92c)
This way we can return properties of the streams other than just their
names in future versions of the API without breaking old clients.
The manual step required is to deploy the updated version of
sync-public-streams on zmirror.humbughq.com when we deploy this code
to prod.
(imported from commit 42b86d8daa5729f52c9961dd912c5776a25ab0b4)
For consistency, and because nobody could think of a reason to have it live
in bots/ with a symlink.
(imported from commit def372653fcdde2805729134fec9d4bc3ce294ec)
Modified files need to be copied into the right place. The checkout
on git.humbughq.com also needs to be updated.
(imported from commit dbe9e05a0512e1f59c7819dd8d44c2c4e9c83bcf)
feedback-bot needs to be updated and restarted after this is pushed to
prod for these changes to take effect.
(imported from commit fcabd2f4bb26c794454e096242a8073805fc786c)
We leave the stuff under api/ alone for now, since we need to be able to ship
it as a standalone thing.
tools/post-receive wasn't using the function anyway.
For push to master: Push this commit, update post-receive per instructions at
the top of that file, then push the rest of the branch to confirm that the hook
still works.
No manual instructions for prod.
(imported from commit 9bcbe14c08d15eda47d82f0b702bad33e217a074)
At Ksplice we used /usr/bin/python because we shipped dependencies as Debian /
Red Hat packages, which would be installed against the system Python. We were
also very careful to use only Python 2.3 features so that even old system
Python would still work.
None of that is true at Humbug. We expect users to install dependencies
themselves, so it's more likely that the Python in $PATH is correct. On OS X
in particular, it's common to have five broken Python installs and there's no
expectation that /usr/bin/python is the right one.
The files which aren't marked executable are not interesting to run as scripts,
so we just remove the line there. (In general it's common to have libraries
that can also be executed, to run test cases or whatever, but that's not the
case here.)
(imported from commit 437d4aee2c6e66601ad3334eefd50749cce2eca6)
Realistically, if the bot crashes once, it'll probably crash the next
time too, so I'm not convinced we need this loop at all, but in the
interests of avoiding churn on an extensively tested script, I'm going
to err on the side of the minimal change here.
(imported from commit e2bbd3700395ba4d0b181a4616e816e8f1231669)
This change adds a copyright notice and moves our site-specific bits
to global constants at the top of the file.
(imported from commit ccc8cf10f2d0d70c7500b12c7849406268313bae)
This adds two characters to the length of our default format field, but
based on a conversation I had with kcr, I think this should probably be
okay. If it's a problem, the symptom we'll see is that certain people
will be unable to send zephyrs with this default format (so, certain
Humbug users will have their forwarding consistently fail).
We need to remember to, in a future commit (once everyone has started
using the updated version), remove the:
> or notice.format.endswith("@(@color(blue))")
(imported from commit 703ef60f524646bca8d5099c9066efabd365be43)
Fixes#602.
I replaced the SIGKILL with a SIGINT, and then catch SIGINT with a
handler. This handler calls cancelSubs if necessary, and can later be
edited to perform other clean-up operations, too. I thought about, in
this same commit, changing the SIGTERM in
maybe_restart_mirroring_script to a SIGINT, but after tracing out the
code paths, I realized that isn't necessary. (The SIGTERM is
necessarily performed on a process that has not subscribed to any
zephyr classes, so cancelSubs is unnecessary. If we do think that we
may want to add additional clean-up operations in the future, though,
then it might be worth investigating changing this SIGTERM.)
(imported from commit 692b295be6cb40b0e4ec2ca0bc58c58056ed9bd9)
Bots are not part of what we distribute, so put them in the repo root.
We also updated some of the bots to use relative path names.
(imported from commit 0471d863450712fd0cdb651f39f32e9041df52ba)
Previously, if users of our code put the API folder in their pyshared
they would have to import it as "humbug.humbug". By moving Humbug's API
into a directory named "humbug" and moving the API into __init__, you
can just "import humbug".
(imported from commit 1d2654ae57f8ecbbfe76559de267ec4889708ee8)
