Tim Abbott
305a8ce569
zephyr_mirror: Run the class mirror in 16 parallel shards.
...
(imported from commit cb17ef999e94cc57d5b01114479728b04475c9c4)
2012-11-27 12:09:57 -05:00
Tim Abbott
bfec56661e
zephyr_mirror: Add a --no-forward-from-humbug option.
...
We'll use this to only have one of our shards forward from Humbug.
(imported from commit a6f3c2180b49c1fa826baf1ae9e7e3ba252a5bfa)
2012-11-27 12:09:57 -05:00
Tim Abbott
975c8ef5b0
zephyr_mirror: Clean up unnecessary logic in process_notice.
...
(imported from commit 28599789016fd1006c2de4d5ddc3cf1f10baa47f)
2012-11-27 12:09:57 -05:00
Tim Abbott
dbeb107aaf
zephyr_mirror: Fix the check for whether a Zephyr is a personal.
...
(imported from commit 609b6339fa82ecbd6435a386d24bf555e75a8120)
2012-11-27 12:09:57 -05:00
Tim Abbott
b081d2fa5a
zephyr_mirror: Reorder process_notice to log fewer "Skipping" messages.
...
Previously, we were sending "Skipping message we got from Humbug!"
for messages we wouldn't have forwarded anyway.
(imported from commit 36df85a61336ac00e3d7913d5a417d6b42764350)
2012-11-27 12:09:57 -05:00
Tim Abbott
f45a8bebf2
zephyr_mirror: Don't display/log some useless "Skipping" notices.
...
In this case, if we're configured to not forward personals, there's no
point in logging a decision not to forward one.
(imported from commit 62c37591c6a70afb6235de626b0c6a3502cbcb27)
2012-11-27 12:09:56 -05:00
Tim Abbott
333573f06b
zephyr_mirror: Simplify optparse usage.
...
(imported from commit d7cce4236dae6f369ea1cc4e12f89c2011718b7f)
2012-11-27 12:09:56 -05:00
Tim Abbott
ca34a2a02f
check-mirroring: Fix some issues with startup process timing.
...
It seems that check-mirroring was reporting a lot of spurious failures
due to it sometimes taking more than 3 seconds for the setup phase of
check-mirroring's receive path to run. So fix this:
(1) Wait a bit more than 3 seconds for the receiver to subscribe to
messages
(2) Subscribe to Humbug messages before forking (we can't do this with
zephyr.init() because python-zephyr gets totally messed up if you use
it from multiple processes with a shared initialization)
(3) Get rid of the old time.sleep(0.x) values that were intended to
make messages arrive in order -- since we're now checking that
messages correctly arrived using set(), they aren't needed.
(4) Use a single request to subscribe to both zephyr classes we need
to subscribe to (saves 1 RTT).
(imported from commit d96aef05405ce43e9a4a549de189da9a2e393875)
2012-11-26 17:10:15 -05:00
Tim Abbott
ba33df97bb
zephyr_mirror: Don't use @(@color(blue)) in zsigs.
...
(imported from commit e4b594e3c22c25fa4f87baaad09227396bb92a0d)
2012-11-26 16:19:09 -05:00
Tim Abbott
135f7f1b57
zephyr_mirror: Use default format to indicate what came from Humbug.
...
(imported from commit 5c2fdccb08d605a37d6c87c37f023c06a0a9a760)
2012-11-26 16:18:05 -05:00
Tim Abbott
111a9bc0e6
zephyr_mirror: Refactor zwrite_args handling a bit.
...
(imported from commit 4923c90dc546022d48f226c2a204e9d09717b563)
2012-11-26 14:58:34 -05:00
Keegan McAllister
84d73db4f3
Send Nagios alerts to Humbug
...
Fixes #385 .
(imported from commit 7dac013debd6ccff031fc4da0dd7185e198b4498)
2012-11-26 14:42:55 -05:00
Tim Abbott
fcf24e30f5
check-mirroring: Add subscriptions inside IOError check.
...
(imported from commit d3b00ce9d91a0bcffcef58436a4b90770155fce9)
2012-11-26 12:25:39 -05:00
Tim Abbott
9d6ce4ee0b
api: Add copyright and MIT license notices to distributed tools.
...
(imported from commit 9f93050f988549dc363f5a7c798c8a9fa1c01d29)
2012-11-26 12:25:38 -05:00
Tim Abbott
27d597d686
zephyr_mirror: Don't break long words/urls when mirroring.
...
(imported from commit df2b6a6969fc54016eeee4996df317e2843844c8)
2012-11-21 15:55:18 -05:00
Tim Abbott
27b02176bc
zephyr_mirror: Fix running --sync-subscriptions repeatedly.
...
(imported from commit 15bead790590c91cd4a1d794799f94341d31c181)
2012-11-20 17:17:40 -05:00
Jessica McKellar
34abe48f62
nagios: Increase the timeout on MIT zephyr forwarding functioning.
...
Sometimes messages are delayed passed the allowed 10 seconds. We may
eventually decide that that amount of latency is unacceptable, but the
latency is not what this script is supposed to be checking.
(imported from commit d83a6a83d60e9eac13b3b87fb31de7f9881acabf)
2012-11-20 15:32:06 -05:00
Tim Abbott
244ff8459e
zephyr_mirror: Add exception handling around all main loop code.
...
(imported from commit 5dc46decf882302a6c58f3bb788362f82f293b35)
2012-11-19 12:44:12 -05:00
Tim Abbott
36280c2769
zephyr_mirror: Use minimal wrapper script that is hard to accidentally break.
...
(imported from commit 45967b8b806df616b0480b217dd5db82c5a2cabd)
2012-11-19 12:44:06 -05:00
Tim Abbott
70e7c9fd14
zephyr_mirror: Move parse_args into its own function.
...
(imported from commit b56ca627e3d34b8dcefead2cfecfa77bb428b0ec)
2012-11-19 12:30:15 -05:00
Tim Abbott
99f90762d3
zephyr_mirror: Get rid of extra_mirror code -- it is no longer used.
...
(imported from commit ab2a9cfcfa6a8c7615523ebb16864e9a4db05f3a)
2012-11-19 12:11:07 -05:00
Tim Abbott
ff80f7df71
zephyr_mirror: Improve robustness when killing child processes.
...
(imported from commit f212b87ffb1fc814d289c14d3854fb1f2f6e2cf3)
2012-11-19 11:46:05 -05:00
Tim Abbott
ce279d48ae
zephyr_mirror: Fix restart_stamp functionality.
...
(imported from commit a79db110d3372ab88fabfc0b7013c5fd3ad07f8e)
2012-11-19 11:20:33 -05:00
Tim Abbott
5411e5c360
zephyr_mirror: Improve zwrite error handling.
...
(imported from commit a6b5b9c0c9bd6ffde0b2b783c87528bdcae03566)
2012-11-16 17:08:51 -05:00
Tim Abbott
584743eca8
[manual] Uniformize the subscriptions API method names.
...
This requires a zephyr_mirror deployment when it is pushed to prod.
(imported from commit 6543441fb264b518f8705d7989d068a1d50ce5d6)
2012-11-16 17:06:01 -05:00
Jessica McKellar
5b285f2321
nagios: print more information on check-mirroring errors.
...
(imported from commit 6ea149d939ae309745ce8c5b2ab7ae2237007d97)
2012-11-16 11:25:22 -05:00
Tim Abbott
6727e9dbd9
[manual] send_message: Rename recipient/stream fields to 'to'.
...
This commit changes APIs and requires and update of all zephyr
mirroring bots to deploy properly.
(imported from commit 2672d2d07269379f7a865644aaeb6796d54183e1)
2012-11-15 15:30:06 -05:00
Tim Abbott
fb59f15099
check-mirroring: Use consistent ordering for keys.
...
(imported from commit 7deaf55ccb4b47a11dbd30c9239893a42eb39e00)
2012-11-15 14:56:10 -05:00
Tim Abbott
4ca1572283
zephyr_mirror: Improve error handling getting public streams list.
...
(imported from commit 06f9c00f19056bfe7db3c02955e3a29ad270de6e)
2012-11-15 14:51:05 -05:00
Tim Abbott
c751cc97db
zephyr_mirror: Send Humbugs in child processes.
...
This improves the throughput of mirroring a large number of zephyrs in
a row from about 1.5/second to about 9/second, which is basically
satisfactory.
(imported from commit 5f72680d6290eaa02ef8ced5b3792fb3efc1db41)
2012-11-15 14:50:59 -05:00
Waseem Daher
ca351f3ddc
Ask user to reload Humbug after running sync-subscriptions.
...
(imported from commit d374d5b8eefd4240b8b11f83080fcdf2ff3cadaf)
2012-11-15 10:40:10 -05:00
Jessica McKellar
c571c997b3
If a zwrite fails, retry unauthenticated and warn the user.
...
(imported from commit a8a516d59f4af72ed2eaebb6bfa65e2f359fc2a6)
2012-11-14 17:38:42 -05:00
Tim Abbott
7514390190
zephyr_mirror: Don't use the /mit symlink for use on systems without automounters.
...
(imported from commit b376e6da6c971d887f2d75194a8ed0d3adaddfea)
2012-11-14 15:03:49 -05:00
Tim Abbott
e182bfcb98
zephyr_mirror: Don't try subscribing to an empty stream list.
...
(imported from commit c8155459b768dd1fc192246c1f584abb30a7bc02)
2012-11-14 15:03:49 -05:00
Tim Abbott
1a5f0648c5
zephyr_mirror: Don't send fullname data to server.
...
(imported from commit 743d0109f422fea5c4642e83448cd179f4e60827)
2012-11-14 15:03:49 -05:00
Keegan McAllister
e42097a723
gcal-bot: Get API key from ~/.humbug-api-key
...
(imported from commit 63d63ad0d299c300d28de61069ada8ea2c8eb839)
2012-11-14 10:38:36 -05:00
Keegan McAllister
11cdc959e6
gcal-bot: Fix sys.path for new location under api/bots/
...
(imported from commit 9f5b90d5e0cd2134f2f0210980ad49fd8062a9c4)
2012-11-14 10:38:36 -05:00
Zev Benjamin
1a4f97a874
zephyr_mirror: forward messages in test mode if the site is not the default one
...
(imported from commit 7d504a6d555866f12f1657b522b4e6eb380c64a3)
2012-11-13 20:22:37 -05:00
Zev Benjamin
73daa73887
Unify huddles and personals into private messages on the send path
...
Personals are now just private messages between two people (which
sometimes manifests as a private message with one recipient). The
new message type on the send path is 'private'. Note that the receive
path still has 'personal' and 'huddle' message types.
(imported from commit 97a438ef5c0b3db4eb3e6db674ea38a081265dd3)
2012-11-13 15:40:53 -05:00
Zev Benjamin
8a76560970
Use a JSON array for recipients in send_message
...
(imported from commit e2184f92b708cc2e8ef3e9ae79ee4241c0aa12a1)
2012-11-13 15:40:53 -05:00
Tim Abbott
4c2d56c10b
zephyr_mirror: Extend logging on subscribe failures.
...
(imported from commit ce54549a74935bb95cb68dd1e6db2a3eec6166c5)
2012-11-13 12:47:59 -05:00
Tim Abbott
dd07236eec
zephyr_mirror: Tweak linewrapping algorithm.
...
(imported from commit 1165a7680251f6bb365f44a60612fffe0fbbae42)
2012-11-13 12:43:37 -05:00
Tim Abbott
defaeb935e
zephyr_mirror: Use python logging module.
...
(imported from commit 3cd9506d46eb56fbd5a48c789160a45846332633)
2012-11-13 12:43:14 -05:00
Tim Abbott
c38587191e
zephyr_mirror: Support whitespace-only zephyr instances.
...
Previously messages sent to zephyr instance " " wouldn't be forwarded
zephyr=>humbug because the target instance is the null instance; I
learned this when we got a few tracebacks in the zephyr_mirror log, so
clearly this does happen.
(imported from commit 08bd7470e75ac6af24ac83696b6cf68d70654664)
2012-11-09 17:32:15 -05:00
Tim Abbott
2735c8410e
zephyr_mirror: Catch IOError from zephyr.init().
...
(imported from commit d657fede53ee3754e76edf66b10bef09ad1f9577)
2012-11-09 16:56:14 -05:00
Tim Abbott
35fbf350ae
Fix sending to cross-realm Zephyr users.
...
(imported from commit 1eebf286e149aa1242d6d26096003eb696fb18f2)
2012-11-09 16:56:13 -05:00
Tim Abbott
c3c3045e3d
zephyr_mirror: Forward messages sent to the empty instance more clearly.
...
(imported from commit 1fb5e4f8ce9941b3ca89688821d325d7ae49aee5)
2012-11-08 17:48:15 -05:00
Tim Abbott
8d27bcbad7
zephyr_mirror: Add a test mode that just prints messages.
...
This is useful for trying out new versions of the forwarding code
while the mirroring bot is still running in production.
(imported from commit bcdaf91fed55ac0974b1efe31dd13ed006b6fd06)
2012-11-08 17:48:15 -05:00
Tim Abbott
ef065d112f
check-mirroring: Move some prints to be conditional on "verbose".
...
(imported from commit 90db591f5fc3e4d1f830ed16bca9380a3a9dddd3)
2012-11-08 13:47:02 -05:00
Tim Abbott
6aad2946a1
check-mirroring: Make root_path and option (with new default).
...
(imported from commit df15d089f234bfe5373fc3fed95eb2f3b3ee42b2)
2012-11-08 13:34:32 -05:00