Skip to content

Instantly share code, notes, and snippets.

@bowrocker
Created June 18, 2020 17:26
Show Gist options
  • Save bowrocker/b8fe02363ba9bc4b3dd52702b1f56fae to your computer and use it in GitHub Desktop.
Save bowrocker/b8fe02363ba9bc4b3dd52702b1f56fae to your computer and use it in GitHub Desktop.
(base) joyentmac12398:manta-rebalancer jonran$ sudo make test
Password:
RUST_LOG=remora=trace cargo test agenttests
Finished test [unoptimized + debuginfo] target(s) in 1.02s
Running target/debug/deps/rebalancer_agent-22956bc1022123e6
running 6 tests
{"msg":"listening","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.793676-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0","address":"127.0.0.1:8878"}
{"msg":"rebalancer[ThreadId(22):tokio-runtime-worker-3]: Received assignment a6d94808-f8d0-443d-b80d-cdabb682016f.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.804909-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(22):tokio-runtime-worker-3]: Received assignment: RwLock {\n data: Assignment {\n uuid: \"a6d94808-f8d0-443d-b80d-cdabb682016f\",\n stats: AgentAssignmentStats {\n state: Scheduled,\n failed: 0,\n complete: 0,\n total: 1,\n },\n tasks: [\n Task {\n object_id: \"area_codes_by_state.csv\",\n owner: \"rebalancer\",\n md5sum: \"QXBlX0QFcscVIwptkUaI8g==\",\n source: MantaObjectShark {\n datacenter: \"dc\",\n manta_storage_id: \"localhost:8080\",\n },\n status: Pending,\n },\n ],\n },\n}","v":0,"name":"rebalancer","level":20,"time":"2020-06-18T13:23:07.805062-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(4):agenttests::duplicate_assignment]: Response: \"a6d94808-f8d0-443d-b80d-cdabb682016f\"","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.808594-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Begin processing assignment a6d94808-f8d0-443d-b80d-cdabb682016f.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.809588-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Processing task: assignment: a6d94808-f8d0-443d-b80d-cdabb682016f, owner: rebalancer, object: area_codes_by_state.csv","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.809743-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(21):tokio-runtime-worker-2]: Received assignment 164fda5c-5bac-4cc0-b6ae-168f61e7853b.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.813250-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(21):tokio-runtime-worker-2]: Received assignment: RwLock {\n data: Assignment {\n uuid: \"164fda5c-5bac-4cc0-b6ae-168f61e7853b\",\n stats: AgentAssignmentStats {\n state: Scheduled,\n failed: 0,\n complete: 0,\n total: 1,\n },\n tasks: [\n Task {\n object_id: \"area_codes_by_state.csv\",\n owner: \"rebalancer\",\n md5sum: \"QXBlX0QFcscVIwptkUaI8g==\",\n source: MantaObjectShark {\n datacenter: \"dc\",\n manta_storage_id: \"localhost:8080\",\n },\n status: Pending,\n },\n ],\n },\n}","v":0,"name":"rebalancer","level":20,"time":"2020-06-18T13:23:07.813353-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(2):agenttests::delete_assignment]: Response: \"164fda5c-5bac-4cc0-b6ae-168f61e7853b\"","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.815648-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Download response for http://localhost:8080/rebalancer/area_codes_by_state.csv is 200 OK","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.817483-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: owner: rebalancer, object: area_codes_by_state.csv, bytes: 4662","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.819662-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Finished processing assignment a6d94808-f8d0-443d-b80d-cdabb682016f.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.820082-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(22):tokio-runtime-worker-3]: Received assignment 5e340a6b-e6aa-4feb-94cd-7c7543f268d7.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.820259-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(22):tokio-runtime-worker-3]: Received assignment: RwLock {\n data: Assignment {\n uuid: \"5e340a6b-e6aa-4feb-94cd-7c7543f268d7\",\n stats: AgentAssignmentStats {\n state: Scheduled,\n failed: 0,\n complete: 0,\n total: 1,\n },\n tasks: [\n Task {\n object_id: \"area_codes_by_state.csv\",\n owner: \"rebalancer\",\n md5sum: \"QXBlX0QFcscVIwptkUaI8g==\",\n source: MantaObjectShark {\n datacenter: \"dc\",\n manta_storage_id: \"localhost:8080\",\n },\n status: Pending,\n },\n ],\n },\n}","v":0,"name":"rebalancer","level":20,"time":"2020-06-18T13:23:07.820357-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Begin processing assignment 164fda5c-5bac-4cc0-b6ae-168f61e7853b.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.821868-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Processing task: assignment: 164fda5c-5bac-4cc0-b6ae-168f61e7853b, owner: rebalancer, object: area_codes_by_state.csv","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.822001-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(7):agenttests::replace_healthy]: Response: \"5e340a6b-e6aa-4feb-94cd-7c7543f268d7\"","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.822209-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Checksum passed -- no need to download: rebalancer/area_codes_by_state.csv","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.824096-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Finished processing assignment 164fda5c-5bac-4cc0-b6ae-168f61e7853b.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.824182-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Begin processing assignment 5e340a6b-e6aa-4feb-94cd-7c7543f268d7.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.826326-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(23):tokio-runtime-worker-4]: Received assignment ecc7de15-0a98-4dbd-9a5b-a28c8d382791.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.826424-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Processing task: assignment: 5e340a6b-e6aa-4feb-94cd-7c7543f268d7, owner: rebalancer, object: area_codes_by_state.csv","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.826547-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(23):tokio-runtime-worker-4]: Received assignment: RwLock {\n data: Assignment {\n uuid: \"ecc7de15-0a98-4dbd-9a5b-a28c8d382791\",\n stats: AgentAssignmentStats {\n state: Scheduled,\n failed: 0,\n complete: 0,\n total: 1,\n },\n tasks: [\n Task {\n object_id: \"abc\",\n owner: \"rebalancer\",\n md5sum: \"QXBlX0QFcscVIwptkUaI8g==\",\n source: MantaObjectShark {\n datacenter: \"dc\",\n manta_storage_id: \"localhost:8080\",\n },\n status: Pending,\n },\n ],\n },\n}","v":0,"name":"rebalancer","level":20,"time":"2020-06-18T13:23:07.826681-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Checksum passed -- no need to download: rebalancer/area_codes_by_state.csv","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.826996-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Finished processing assignment 5e340a6b-e6aa-4feb-94cd-7c7543f268d7.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.827132-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(6):agenttests::object_not_found]: Response: \"ecc7de15-0a98-4dbd-9a5b-a28c8d382791\"","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.829086-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Begin processing assignment ecc7de15-0a98-4dbd-9a5b-a28c8d382791.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.830190-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Processing task: assignment: ecc7de15-0a98-4dbd-9a5b-a28c8d382791, owner: rebalancer, object: abc","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.830279-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Download response for http://localhost:8080/rebalancer/abc is 404 Not Found","v":0,"name":"rebalancer","level":50,"time":"2020-06-18T13:23:07.832030-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(23):tokio-runtime-worker-4]: Received assignment 51e23ab5-6c5f-453a-976d-0290ee03bace.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.832125-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(23):tokio-runtime-worker-4]: Received assignment: RwLock {\n data: Assignment {\n uuid: \"51e23ab5-6c5f-453a-976d-0290ee03bace\",\n stats: AgentAssignmentStats {\n state: Scheduled,\n failed: 0,\n complete: 0,\n total: 1,\n },\n tasks: [\n Task {\n object_id: \"area_codes_by_state.csv\",\n owner: \"rebalancer\",\n md5sum: \"abc\",\n source: MantaObjectShark {\n datacenter: \"dc\",\n manta_storage_id: \"localhost:8080\",\n },\n status: Pending,\n },\n ],\n },\n}","v":0,"name":"rebalancer","level":20,"time":"2020-06-18T13:23:07.832245-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Finished processing assignment ecc7de15-0a98-4dbd-9a5b-a28c8d382791.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.832332-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(5):agenttests::failed_checksum]: Response: \"51e23ab5-6c5f-453a-976d-0290ee03bace\"","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.834388-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Begin processing assignment 51e23ab5-6c5f-453a-976d-0290ee03bace.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.835324-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Processing task: assignment: 51e23ab5-6c5f-453a-976d-0290ee03bace, owner: rebalancer, object: area_codes_by_state.csv","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.835466-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Download response for http://localhost:8080/rebalancer/area_codes_by_state.csv is 200 OK","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.837055-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(19):tokio-runtime-worker-0]: Received assignment f1c56495-3903-4b9d-9cf2-dd1cbec83d84.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.837893-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(19):tokio-runtime-worker-0]: Received assignment: RwLock {\n data: Assignment {\n uuid: \"f1c56495-3903-4b9d-9cf2-dd1cbec83d84\",\n stats: AgentAssignmentStats {\n state: Scheduled,\n failed: 0,\n complete: 0,\n total: 1,\n },\n tasks: [\n Task {\n object_id: \"area_codes_by_state.csv\",\n owner: \"rebalancer\",\n md5sum: \"QXBlX0QFcscVIwptkUaI8g==\",\n source: MantaObjectShark {\n datacenter: \"dc\",\n manta_storage_id: \"localhost:8080\",\n },\n status: Pending,\n },\n ],\n },\n}","v":0,"name":"rebalancer","level":20,"time":"2020-06-18T13:23:07.837984-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Checksum failed for rebalancer/area_codes_by_state.csv.","v":0,"name":"rebalancer","level":50,"time":"2020-06-18T13:23:07.838775-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Finished processing assignment 51e23ab5-6c5f-453a-976d-0290ee03bace.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.839137-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(3):agenttests::download]: Response: \"f1c56495-3903-4b9d-9cf2-dd1cbec83d84\"","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.839962-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Begin processing assignment f1c56495-3903-4b9d-9cf2-dd1cbec83d84.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.842021-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Processing task: assignment: f1c56495-3903-4b9d-9cf2-dd1cbec83d84, owner: rebalancer, object: area_codes_by_state.csv","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:07.842143-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Checksum passed -- no need to download: rebalancer/area_codes_by_state.csv","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.842509-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Finished processing assignment f1c56495-3903-4b9d-9cf2-dd1cbec83d84.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:07.842612-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(22):tokio-runtime-worker-3]: Assignment a6d94808-f8d0-443d-b80d-cdabb682016f has already been received.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:37.830322-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
test agenttests::duplicate_assignment ... ok
test agenttests::delete_assignment ... ok
{"msg":"rebalancer[ThreadId(24):tokio-runtime-worker-5]: Received assignment 3a2be6c2-02b8-4262-a901-7169b75e891e.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:37.842326-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(24):tokio-runtime-worker-5]: Received assignment: RwLock {\n data: Assignment {\n uuid: \"3a2be6c2-02b8-4262-a901-7169b75e891e\",\n stats: AgentAssignmentStats {\n state: Scheduled,\n failed: 0,\n complete: 0,\n total: 1,\n },\n tasks: [\n Task {\n object_id: \"area_codes_by_state.csv\",\n owner: \"rebalancer\",\n md5sum: \"QXBlX0QFcscVIwptkUaI8g==\",\n source: MantaObjectShark {\n datacenter: \"dc\",\n manta_storage_id: \"localhost:8080\",\n },\n status: Pending,\n },\n ],\n },\n}","v":0,"name":"rebalancer","level":20,"time":"2020-06-18T13:23:37.842462-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
test agenttests::object_not_found ... ok
test agenttests::failed_checksum ... ok
{"msg":"rebalancer[ThreadId(7):agenttests::replace_healthy]: Response: \"3a2be6c2-02b8-4262-a901-7169b75e891e\"","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:37.845562-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Begin processing assignment 3a2be6c2-02b8-4262-a901-7169b75e891e.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:37.846509-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Processing task: assignment: 3a2be6c2-02b8-4262-a901-7169b75e891e, owner: rebalancer, object: area_codes_by_state.csv","v":0,"name":"rebalancer","level":10,"time":"2020-06-18T13:23:37.846614-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(18)]: Checksum passed -- no need to download: rebalancer/area_codes_by_state.csv","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:37.846939-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
{"msg":"rebalancer[ThreadId(11)]: Finished processing assignment 3a2be6c2-02b8-4262-a901-7169b75e891e.","v":0,"name":"rebalancer","level":30,"time":"2020-06-18T13:23:37.847048-04:00","hostname":"joyentmac12398.local","pid":75329,"build-id":"0.1.0"}
test agenttests::download ... ok
test agenttests::replace_healthy ... test agenttests::replace_healthy has been running for over 60 seconds
test agenttests::replace_healthy ... ok
test result: ok. 6 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
Running target/debug/deps/manager-4d9e50a4a6fb89e1
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 17 filtered out
Running target/debug/deps/rebalancer_adm-c00f7310d2b2184e
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 5 filtered out
Running target/debug/deps/rebalancer_manager-75cd9c6b170d7ac6
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 3 filtered out
Running target/debug/deps/rebalancer-00a2fe60062650e0
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
RUST_LOG=remora=trace cargo test job -- --test-threads=1
Finished test [unoptimized + debuginfo] target(s) in 0.24s
Running target/debug/deps/rebalancer_agent-22956bc1022123e6
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 6 filtered out
Running target/debug/deps/manager-4d9e50a4a6fb89e1
running 12 tests
test jobs::evacuate::tests::assignment_processing_test ... error: test failed, to rerun pass '-p manager --lib'
make: *** [jobtests] Error 1
(base) joyentmac12398:manta-rebalancer jonran$
(base)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment