Commit graph

42 commits

Author SHA1 Message Date
Tim Abbott f65b130b45 zmirror: Fix printing of list of stream failures.
(imported from commit 0d573b412809263dbbacdf1b475adac440dae352)
2013-01-02 10:22:08 -05:00
Tim Abbott 2919ce2544 zephyr_mirror: Cancel Zephyr subscriptions on graceful restart.
(imported from commit 39737bbf7ea76181ab25a0eaacf230906cba3d87)
2012-12-21 12:44:19 -05:00
Tim Abbott 25bb86d2d0 zephyr_mirror: Fix killing duplicate zephyr_mirror processes.
The refactoring that we did a couple weeks ago to make the zephyr
mirror script restart itself automatically (by splitting it into
zephyr_mirror.py and zephyr_mirror_backend.py) had a poor interaction
with our code for killing old zephyr_mirror processes (to prevent
double-mirroring).  If you manually ran two copies of the outer
mirroring script (zephyr_mirror.py), then the second one would on
startup kill the first one's zephyr_mirror_backend.py children (for
being duplicate zephyr_mirror_backend.py processes that would result
in double-mirroring).  However, importantly, it did not kill the first
mirroring script's zephyr_mirror.py parent process, so the first
mirroring script would then proceed to startup up new children.  The
process then repeats, with the two scripts' roles reversed.

This issue didn't affect the sharded mirroring case, where I had been
doing the testing of the kill code with that refactoring, because we
don't have a version of the outer zephyr_mirror.py loop for that
situation (a consequence of it being hard to restart the threads
properly with the run_parallel API that we're using to spawn all the
children).

(imported from commit d4886ac77312a6b0ebd0d612f6fb084970bf23a2)
2012-12-21 12:44:19 -05:00
Tim Abbott a22f94633b zephyr_mirror: Fix insecure temporary file for log.
This is probably a lot more annoying to use, in that e.g. there are
separate log files for the two directions of the mirror, but we
haven't used these logs for much, so whatever.

(imported from commit d3bc407d90099214d242526c01cd3d3cd9d9d9bd)
2012-12-20 14:30:20 -05:00
Tim Abbott e609692e33 zephyr_mirror: Add a few more restricted-access classes.
(imported from commit 6690e5dcd6a313ac4e4ef2351e300306ca7fb3c5)
2012-12-19 08:49:14 -05:00
Waseem Daher a3fba04fdd Don't consider hyphens to be an acceptable place to wordwrap.
Otherwise, URLs like this one from catherio:
  http://www.sadiemagazine.com/issue-no-11/arts-letters/comic/i-think-i-am-in-friend-love-with-you
don't go to zephyr as one line, which makes them really
annoying to click on barnowl, etc.

(imported from commit d14f0af7d24a6cb8bc7cb0582f60116308f59923)
2012-12-18 11:03:18 -05:00
Tim Abbott 08f165e3ac zephyr_mirror: Add comment explaining why we drop the @ATHENA.MIT.EDU in zwrite -C.
(imported from commit b591e2e0d6c607ef04e3c0005091078cfe23e3ff)
2012-12-10 17:05:04 -05:00
Tim Abbott 8e0aabc730 zephyr_mirror: Improve debugging output on outgoing messages.
Previously all that we logged was the sender, which turns out to be a
constant for humbug=>zephyr mirroring.

(imported from commit 527a3ac4b9b815a2b4d6b63c3ad92d9d5ad99a6e)
2012-12-10 17:05:04 -05:00
Tim Abbott bb80b86e25 zephyr_mirror: Fix handling of Humbug=>Zephyr personals.
(imported from commit 149f8aaf9076583499392bcb09a0250b05c3dea6)
2012-12-08 13:05:44 -05:00
Tim Abbott 94a553f0f1 zephyr_mirror: Use a variable for humbug_account_email.
(imported from commit 90da406a5eab67d20b42243e96b8afb3e5b5f342)
2012-12-08 13:00:22 -05:00
Tim Abbott 28d98729a5 zephyr_mirror: Fix detection of personals.
(imported from commit 1fc9050bad36dc15e76f65bdb6532e3e7e601a40)
2012-12-05 12:50:14 -05:00
Tim Abbott 37c837904f Don't send zwrite pings.
(imported from commit 8dd7894eda6315ab52e61489e21ec1d201e86cc9)
2012-12-05 12:04:18 -05:00
Zev Benjamin b945113867 [manual] Unify huddles and personals into private messages on the receive path
feedback-bot and zephyr_mirror will need to be updated and restarted
when this is deployed to prod.

(imported from commit fe2b524424c174bcb1b717a851a5d3815fda3f69)
2012-12-04 18:01:51 -05:00
Keegan McAllister b971ce4683 mirroring: Fix import by putting 'api' path first
(imported from commit b35d962d6456f01b97deeeaa328faa03b65494f8)
2012-12-03 17:50:29 -05:00
Keegan McAllister ac9e3d6483 zephyr_mirror_backend: Rename variable 'humbug'
(imported from commit 2e278e4d2452f750cb874510a762fc9616e7dfb3)
2012-12-03 17:49:27 -05:00
Tim Abbott 24b8a7eccc zephyr_mirror: Fix case issue in subscriptions syncing.
(imported from commit e87ddab3684f4acae940ee831caadbc346458a11)
2012-12-03 16:20:19 -05:00
Keegan McAllister b9325a09fa Rename class HumbugAPI to Client
(imported from commit 4270f31fc5febcd9c444d0d133a1dad3860618f0)
2012-12-03 15:54:54 -05:00
Keegan McAllister f9f8a6e603 Rename api.common to humbug
Fixes #482.

(imported from commit 1bd6a7fd993d8d5e225e0311c288dbce0c369a40)
2012-12-03 15:54:54 -05:00
Tim Abbott abb95fca04 zephyr_mirror: Move class name canonicalization earlier.
We should be canonicalizing stream names to class names in
update_subscriptions_from_humbug, before we even decide which classes
to subscribe to; otherwise deduplication and tracking of which classes
we're already subscribed to won't work.

(imported from commit a751b6fca1022390a087516a0730ff77f13d7edf)
2012-11-29 13:57:06 -05:00
Tim Abbott 8683613e76 zephyr_mirror: Add options to run class mirror without sharding for debugging.
(imported from commit 4fcc403504d93e30aca6046cba9f1fe5342faac6)
2012-11-29 13:57:06 -05:00
Tim Abbott 18bbd02074 zephyr_mirror: Don't leak zephyr_mirror processes on send_humbug failures.
(imported from commit be005018672369683d9e3bdbf144f96f58af6b6c)
2012-11-29 13:57:06 -05:00
Tim Abbott 074316105b zephyr_mirror: Replace bare excepts with 'except Exception'.
(imported from commit 857212e0e5116471e418d37845550108c0a47d2b)
2012-11-29 13:57:05 -05:00
Tim Abbott ab68503fcf zephyr_mirror: Compare zephyr subs from server after normalization.
(imported from commit eea290698e83c7d26f23ab3bb0201914417f0e09)
2012-11-28 22:26:10 -05:00
Tim Abbott c61567b67a zephyr_mirror: Add two more restricted access classes.
(imported from commit 06ff0e19c1522ade018aea66f152fdfa55c18e1d)
2012-11-28 22:26:10 -05:00
Tim Abbott 13ebd229c1 get_updates: Replace the "mirror" argument with checking "client".
(imported from commit bd84c2c7a456de8ea7ec35eaf76b5bf8647ca61a)
2012-11-28 17:41:38 -05:00
Tim Abbott 2b2398b712 zephyr_mirror: Pass an encoded stream to hashlib for sharding.
(imported from commit cc57b3956198b66c9cfc84423434a71b8200d819)
2012-11-28 15:44:35 -05:00
Tim Abbott de14210a4d zephyr_mirror: Validate zephyr subscriptions happened successfully.
(imported from commit 494895b87eb9b555175c9890c5c2046cf4ce40ac)
2012-11-28 15:44:35 -05:00
Tim Abbott 121c3cb9c8 zephyr_mirror: Fix display of personals on non-message classes.
(imported from commit 42ff030d2d78c530428dcede406d46fc71521fef)
2012-11-27 14:12:19 -05:00
Tim Abbott 305a8ce569 zephyr_mirror: Run the class mirror in 16 parallel shards.
(imported from commit cb17ef999e94cc57d5b01114479728b04475c9c4)
2012-11-27 12:09:57 -05:00
Tim Abbott bfec56661e zephyr_mirror: Add a --no-forward-from-humbug option.
We'll use this to only have one of our shards forward from Humbug.

(imported from commit a6f3c2180b49c1fa826baf1ae9e7e3ba252a5bfa)
2012-11-27 12:09:57 -05:00
Tim Abbott 975c8ef5b0 zephyr_mirror: Clean up unnecessary logic in process_notice.
(imported from commit 28599789016fd1006c2de4d5ddc3cf1f10baa47f)
2012-11-27 12:09:57 -05:00
Tim Abbott dbeb107aaf zephyr_mirror: Fix the check for whether a Zephyr is a personal.
(imported from commit 609b6339fa82ecbd6435a386d24bf555e75a8120)
2012-11-27 12:09:57 -05:00
Tim Abbott b081d2fa5a zephyr_mirror: Reorder process_notice to log fewer "Skipping" messages.
Previously, we were sending "Skipping message we got from Humbug!"
for messages we wouldn't have forwarded anyway.

(imported from commit 36df85a61336ac00e3d7913d5a417d6b42764350)
2012-11-27 12:09:57 -05:00
Tim Abbott f45a8bebf2 zephyr_mirror: Don't display/log some useless "Skipping" notices.
In this case, if we're configured to not forward personals, there's no
point in logging a decision not to forward one.

(imported from commit 62c37591c6a70afb6235de626b0c6a3502cbcb27)
2012-11-27 12:09:56 -05:00
Tim Abbott 333573f06b zephyr_mirror: Simplify optparse usage.
(imported from commit d7cce4236dae6f369ea1cc4e12f89c2011718b7f)
2012-11-27 12:09:56 -05:00
Tim Abbott ba33df97bb zephyr_mirror: Don't use @(@color(blue)) in zsigs.
(imported from commit e4b594e3c22c25fa4f87baaad09227396bb92a0d)
2012-11-26 16:19:09 -05:00
Tim Abbott 135f7f1b57 zephyr_mirror: Use default format to indicate what came from Humbug.
(imported from commit 5c2fdccb08d605a37d6c87c37f023c06a0a9a760)
2012-11-26 16:18:05 -05:00
Tim Abbott 111a9bc0e6 zephyr_mirror: Refactor zwrite_args handling a bit.
(imported from commit 4923c90dc546022d48f226c2a204e9d09717b563)
2012-11-26 14:58:34 -05:00
Tim Abbott 9d6ce4ee0b api: Add copyright and MIT license notices to distributed tools.
(imported from commit 9f93050f988549dc363f5a7c798c8a9fa1c01d29)
2012-11-26 12:25:38 -05:00
Tim Abbott 27d597d686 zephyr_mirror: Don't break long words/urls when mirroring.
(imported from commit df2b6a6969fc54016eeee4996df317e2843844c8)
2012-11-21 15:55:18 -05:00
Tim Abbott 244ff8459e zephyr_mirror: Add exception handling around all main loop code.
(imported from commit 5dc46decf882302a6c58f3bb788362f82f293b35)
2012-11-19 12:44:12 -05:00
Tim Abbott 36280c2769 zephyr_mirror: Use minimal wrapper script that is hard to accidentally break.
(imported from commit 45967b8b806df616b0480b217dd5db82c5a2cabd)
2012-11-19 12:44:06 -05:00