Commit graph

77 commits

Author SHA1 Message Date
Tim Abbott 264651c7b4 zephyr_mirror: De-zcrypt messages for which we have the key.
For now we only support the AES encryption type since the DES one is
probably not used anymore.

(imported from commit 222606db9f704917e74159e7d07a110187a236e6)
2013-08-20 15:02:07 -04:00
Tim Abbott 7f626d2bba zephyr_mirror: Handle syncing subs with invite-only streams.
(imported from commit ec8d0c0d13ff0a81fa4a5e1225e94e6dad997356)
2013-08-20 15:02:04 -04:00
Tim Abbott 9d55822cfb zephyr_mirror: Add option to control path to streams file.
This makes it easy to test the class mirror on just a single test
stream.

(imported from commit 3decb1ced54507683f1c0634df14f73e693104ef)
2013-08-20 14:57:13 -04:00
Tim Abbott 084847b0d7 Rename the 'humbug' API module to 'zulip'.
(imported from commit b3a3d7c05459cbb0110cd0fbe2197d779f3a6264)
2013-08-08 10:22:31 -04:00
Tim Abbott 4741258ca4 Change humbug => zulip in zephyr_mirror_backend internal variables.
(imported from commit 7cc5d78bfb5db8c9d5a44ed862da304d9e44ff0e)
2013-08-07 10:00:08 -04:00
Tim Abbott 84aa7d4230 Change Humbug => Zulip in zephyr_mirror states.
(imported from commit 67542cd2ba93ff4b03c99b59645b7d22ed937337)
2013-08-07 10:00:07 -04:00
Tim Abbott 4cd4c6897b Change Humbug => Zulip in text/comments.
(imported from commit 2f9d73431ae40e1b9e9e11bc2f4f62f566ae758a)
2013-08-07 10:00:07 -04:00
Tim Abbott e0c77efb3f Escape @ when mirroring content back to Zephyr.
(imported from commit 0ab58b5e8a30c66fb6aeaccaebbe498092de1cd5)
2013-08-01 14:01:58 -04:00
Tim Abbott 22e5a15bc4 [manual] Use api.zulip.com to access the Humbug API.
This needs to be deployed after the nginx configuration is deployed.

(imported from commit 167a3d9d27595d40883bb400ebcc44c4ad9b6a2f)
2013-07-25 17:27:46 -04:00
Tim Abbott d063efa8a0 More miscellaneous humbughq.com=>zulip.com updates.
(imported from commit 9a3dbc4268f251c2c0b20d8b6a7e1dc672f1b385)
2013-07-25 17:27:45 -04:00
Tim Abbott c9fab96e5d zephyr_mirror: Use Humbug full name as zsig.
(imported from commit 8cd24c4ff6c265084764ab367b46e4cb64e8c796)
2013-07-18 13:58:16 -04:00
Jessica McKellar e64d390c99 zephyr mirror backend: Switch from Humbug to Zulip in user-visible text.
(imported from commit cc4da47373362babb2f5f57493831946b29d32cf)
2013-07-15 17:49:00 -04:00
Tim Abbott 6b67e42d1a zephyr_mirror: you no longer need to reload Humbug after subbing.
(imported from commit 23cb4e5c4b3300953e58c029150d4a980bac626e)
2013-06-28 15:57:29 -04:00
Tim Abbott 5e2979b870 [manual] Change API URLs to be based on api.humbughq.com/api.
This must be deployed after we update our running nginx configuration
to serve api.humbughq.com.

(imported from commit b5c34ebdd595f55eecd6dca6a18a37f105107bd5)
2013-06-28 15:57:27 -04:00
Tim Abbott 58f4ffe088 [manual] API add_subscriptions: Change arguments to support options.
Since in the future we might want requests to add subscriptions to
include things like colors, in_home_view, etc., we're changing the
data format for the add_subscriptions API call to pass each stream as
a dictionary, giving a convenient place to put any added options.

The manual step required here is updating the API version in AFS
available for use with the zephyr_mirror.py system.

(imported from commit 364960cca582a0658f0d334668822045c001b92c)
2013-06-25 16:34:45 -04:00
Tim Abbott c849ae3470 zephyr_mirror: Fix incorrect check for already running mirror bots.
(imported from commit 76e758697ac19362e79d4a4995b576f10d229514)
2013-05-20 23:29:53 -04:00
Zev Benjamin 223ac22a13 Fix use of logging instead of logger
(imported from commit cd25471e842e089d5032b07bcd83c6fe44d03a2d)
2013-05-09 13:50:21 -04:00
Tim Abbott fcb6336642 zephyr_mirror: Use select to determine whether new messages are available.
(imported from commit a91f45aa9f3913f38285c050e574bf562b2af2df)
2013-05-09 10:35:49 -04:00
Tim Abbott 5a0e86c6d4 zephyr_mirror: Use json if simplejson is not available.
(imported from commit 4b5f6ff71067cd2697c864c715afbb3b9dd19302)
2013-05-09 10:35:47 -04:00
Tim Abbott 5af166e063 zephyr_mirror: Limit logging of errors when checking restart stamps.
(imported from commit 194b30d757a5f20d6d20bc8aed9154c6aece9837)
2013-05-09 10:35:47 -04:00
Keegan McAllister 396861feb1 Consistently use #!/usr/bin/env python
At Ksplice we used /usr/bin/python because we shipped dependencies as Debian /
Red Hat packages, which would be installed against the system Python.  We were
also very careful to use only Python 2.3 features so that even old system
Python would still work.

None of that is true at Humbug.  We expect users to install dependencies
themselves, so it's more likely that the Python in $PATH is correct.  On OS X
in particular, it's common to have five broken Python installs and there's no
expectation that /usr/bin/python is the right one.

The files which aren't marked executable are not interesting to run as scripts,
so we just remove the line there.  (In general it's common to have libraries
that can also be executed, to run test cases or whatever, but that's not the
case here.)

(imported from commit 437d4aee2c6e66601ad3334eefd50749cce2eca6)
2013-02-20 16:02:30 -05:00
Tim Abbott 67ee231ff9 zmirror: Add backoff-retry code for the Zephyr=>Humbug initialization.
(imported from commit bde30d87d6dae75bbfdcd49b30a1f08b3c585ec7)
2013-02-15 17:28:59 -05:00
Jessica McKellar cec2dfa52e zephyr mirror: remove unused imports.
(imported from commit 3ba0629f0a9009087b90fb6f795e855b569c2a67)
2013-02-06 14:22:17 -05:00
Jessica McKellar a31f443baa zephyr mirror: update subscription reporting to accomodate new results format.
(imported from commit 94a680d509f727962a07c18bc89edcb1146d8772)
2013-02-06 14:22:17 -05:00
Jacob Hurwitz 4448ba595a Change our default format to be more descriptive
This adds two characters to the length of our default format field, but
based on a conversation I had with kcr, I think this should probably be
okay. If it's a problem, the symptom we'll see is that certain people
will be unable to send zephyrs with this default format (so, certain
Humbug users will have their forwarding consistently fail).

We need to remember to, in a future commit (once everyone has started
using the updated version), remove the:
> or notice.format.endswith("@(@color(blue))")

(imported from commit 703ef60f524646bca8d5099c9066efabd365be43)
2013-02-05 14:27:56 -05:00
Jacob Hurwitz 630b7ae521 zephyr_mirror: Fix logging messages to multiple files under /tmp.
Also fixes the fact that certain output was not logged at all.

(imported from commit 0e63a9ef4105921ad7cbe3f7565ce7832c5db25e)
2013-02-05 14:27:55 -05:00
Jacob Hurwitz 9ebfa84385 zmirror: Allow duplicate zmirror processes to die gracefully.
Fixes #602.

I replaced the SIGKILL with a SIGINT, and then catch SIGINT with a
handler.  This handler calls cancelSubs if necessary, and can later be
edited to perform other clean-up operations, too. I thought about, in
this same commit, changing the SIGTERM in
maybe_restart_mirroring_script to a SIGINT, but after tracing out the
code paths, I realized that isn't necessary. (The SIGTERM is
necessarily performed on a process that has not subscribed to any
zephyr classes, so cancelSubs is unnecessary. If we do think that we
may want to add additional clean-up operations in the future, though,
then it might be worth investigating changing this SIGTERM.)

(imported from commit 692b295be6cb40b0e4ec2ca0bc58c58056ed9bd9)
2013-02-05 14:27:55 -05:00
Tim Abbott bd3bd8ca44 zephyr_mirror: Read list of public streams from sync-public-streams ouput.
(imported from commit c91828534bfed4047c16a98b926335f4beded005)
2013-02-05 14:27:55 -05:00
Luke Faraone d8f13837d9 Move files around in the API.
Bots are not part of what we distribute, so put them in the repo root.

We also updated some of the bots to use relative path names.

(imported from commit 0471d863450712fd0cdb651f39f32e9041df52ba)
2013-02-01 15:52:28 -05:00
Luke Faraone f2f4a2f8bd Move the API into a subdirectory for ease of imports.
Previously, if users of our code put the API folder in their pyshared
they would have to import it as "humbug.humbug". By moving Humbug's API
into a directory named "humbug" and moving the API into __init__, you
can just "import humbug".

(imported from commit 1d2654ae57f8ecbbfe76559de267ec4889708ee8)
2013-01-16 16:55:22 -05:00
Tim Abbott 8704f6f9d0 zmirror: Fix running on 32 bit systems.
(This also needs pyzephyr to be compiled for 32 bit).

(imported from commit 06e33e17be517483213d402dd75ec2ac168395d1)
2013-01-02 14:20:24 -05:00
Tim Abbott a18796961f zephyr_mirror: Use modern default_format that discourages displaying it.
(imported from commit bfb2ffbf4ca7d424ffe54c4b2752378f4854db25)
2013-01-02 14:10:21 -05:00
Tim Abbott 1e91142e38 zmirror: Fix use of print rather than logging.foo in send_zephyr.
(imported from commit 7188f5364a864f2f144fb630d7a8c85d69219c42)
2013-01-02 14:10:21 -05:00
Tim Abbott 44c5315cd7 zmirror: Add another Zephyr class we can't subscribe to.
(imported from commit 6d43c901ecfd01574149efc06c782a49c5d64c9e)
2013-01-02 14:10:21 -05:00
Tim Abbott 1351fb5f8b zmirror: Use shorter default_format string.
This was causing about 10% of the time, personals being forwarded by
tabbott/extra@ATHENA.MIT.EDU to get this error:

zwrite: Field too long for buffer while sending notice to
tabbott/extra@ATHENA.MIT.EDU

We don't fully understand the cause of the problem, but this fixes it.

(imported from commit 22c39a1061cde9ad6973ef07aee7227623a3d47d)
2013-01-02 14:10:21 -05:00
Tim Abbott f65b130b45 zmirror: Fix printing of list of stream failures.
(imported from commit 0d573b412809263dbbacdf1b475adac440dae352)
2013-01-02 10:22:08 -05:00
Tim Abbott 2919ce2544 zephyr_mirror: Cancel Zephyr subscriptions on graceful restart.
(imported from commit 39737bbf7ea76181ab25a0eaacf230906cba3d87)
2012-12-21 12:44:19 -05:00
Tim Abbott 25bb86d2d0 zephyr_mirror: Fix killing duplicate zephyr_mirror processes.
The refactoring that we did a couple weeks ago to make the zephyr
mirror script restart itself automatically (by splitting it into
zephyr_mirror.py and zephyr_mirror_backend.py) had a poor interaction
with our code for killing old zephyr_mirror processes (to prevent
double-mirroring).  If you manually ran two copies of the outer
mirroring script (zephyr_mirror.py), then the second one would on
startup kill the first one's zephyr_mirror_backend.py children (for
being duplicate zephyr_mirror_backend.py processes that would result
in double-mirroring).  However, importantly, it did not kill the first
mirroring script's zephyr_mirror.py parent process, so the first
mirroring script would then proceed to startup up new children.  The
process then repeats, with the two scripts' roles reversed.

This issue didn't affect the sharded mirroring case, where I had been
doing the testing of the kill code with that refactoring, because we
don't have a version of the outer zephyr_mirror.py loop for that
situation (a consequence of it being hard to restart the threads
properly with the run_parallel API that we're using to spawn all the
children).

(imported from commit d4886ac77312a6b0ebd0d612f6fb084970bf23a2)
2012-12-21 12:44:19 -05:00
Tim Abbott a22f94633b zephyr_mirror: Fix insecure temporary file for log.
This is probably a lot more annoying to use, in that e.g. there are
separate log files for the two directions of the mirror, but we
haven't used these logs for much, so whatever.

(imported from commit d3bc407d90099214d242526c01cd3d3cd9d9d9bd)
2012-12-20 14:30:20 -05:00
Tim Abbott e609692e33 zephyr_mirror: Add a few more restricted-access classes.
(imported from commit 6690e5dcd6a313ac4e4ef2351e300306ca7fb3c5)
2012-12-19 08:49:14 -05:00
Waseem Daher a3fba04fdd Don't consider hyphens to be an acceptable place to wordwrap.
Otherwise, URLs like this one from catherio:
  http://www.sadiemagazine.com/issue-no-11/arts-letters/comic/i-think-i-am-in-friend-love-with-you
don't go to zephyr as one line, which makes them really
annoying to click on barnowl, etc.

(imported from commit d14f0af7d24a6cb8bc7cb0582f60116308f59923)
2012-12-18 11:03:18 -05:00
Tim Abbott 08f165e3ac zephyr_mirror: Add comment explaining why we drop the @ATHENA.MIT.EDU in zwrite -C.
(imported from commit b591e2e0d6c607ef04e3c0005091078cfe23e3ff)
2012-12-10 17:05:04 -05:00
Tim Abbott 8e0aabc730 zephyr_mirror: Improve debugging output on outgoing messages.
Previously all that we logged was the sender, which turns out to be a
constant for humbug=>zephyr mirroring.

(imported from commit 527a3ac4b9b815a2b4d6b63c3ad92d9d5ad99a6e)
2012-12-10 17:05:04 -05:00
Tim Abbott bb80b86e25 zephyr_mirror: Fix handling of Humbug=>Zephyr personals.
(imported from commit 149f8aaf9076583499392bcb09a0250b05c3dea6)
2012-12-08 13:05:44 -05:00
Tim Abbott 94a553f0f1 zephyr_mirror: Use a variable for humbug_account_email.
(imported from commit 90da406a5eab67d20b42243e96b8afb3e5b5f342)
2012-12-08 13:00:22 -05:00
Tim Abbott 28d98729a5 zephyr_mirror: Fix detection of personals.
(imported from commit 1fc9050bad36dc15e76f65bdb6532e3e7e601a40)
2012-12-05 12:50:14 -05:00
Tim Abbott 37c837904f Don't send zwrite pings.
(imported from commit 8dd7894eda6315ab52e61489e21ec1d201e86cc9)
2012-12-05 12:04:18 -05:00
Zev Benjamin b945113867 [manual] Unify huddles and personals into private messages on the receive path
feedback-bot and zephyr_mirror will need to be updated and restarted
when this is deployed to prod.

(imported from commit fe2b524424c174bcb1b717a851a5d3815fda3f69)
2012-12-04 18:01:51 -05:00
Keegan McAllister b971ce4683 mirroring: Fix import by putting 'api' path first
(imported from commit b35d962d6456f01b97deeeaa328faa03b65494f8)
2012-12-03 17:50:29 -05:00
Keegan McAllister ac9e3d6483 zephyr_mirror_backend: Rename variable 'humbug'
(imported from commit 2e278e4d2452f750cb874510a762fc9616e7dfb3)
2012-12-03 17:49:27 -05:00