Skip to content

Instantly share code, notes, and snippets.

@kemingy
Last active April 4, 2022 13:20
Show Gist options
  • Save kemingy/5de644c21e58f48cd706e9be5af5f872 to your computer and use it in GitHub Desktop.
Save kemingy/5de644c21e58f48cd706e9be5af5f872 to your computer and use it in GitHub Desktop.
from flask import Flask, request
app = Flask(__name__)
@app.route("/inference", methods=["POST"])
def inference():
return request.get_data()
if __name__ == "__main__":
app.run(port=8000)
from mosec import Server, Worker
class Hello(Worker):
def forward(self, data):
return data
if __name__ == "__main__":
server = Server()
server.append_worker(Hello)
server.run()

Just for fun

The test.json file only contains {"msg": "ping"}.

Tested with

  • Python v3.9.1
  • macOS 11.6.4 (16-inch 2019 MacBook Pro)
  • CPU 2.6 GHz

Result

  • flask
echo "POST http://localhost:8000/inference" | ./vegeta attack -duration=5s -rate=0 -max-workers=100 -body test.json | tee results.bin | ./
vegeta report
Requests      [total, rate, throughput]         3122, 624.16, 610.59
Duration      [total, attack, wait]             5.113s, 5.002s, 111.206ms
Latencies     [min, mean, 50, 90, 95, 99, max]  10.151ms, 161.981ms, 150.001ms, 198.953ms, 315.383ms, 383.486ms, 439.8ms
Bytes In      [total, mean]                     81172, 26.00
Bytes Out     [total, mean]                     81172, 26.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:3122
Error Set:
echo "POST http://localhost:8000/inference" | ./vegeta attack -duration=5s -rate=0 -max-workers=100 -body test.json | tee results.bin | ./
vegeta report
Requests      [total, rate, throughput]         37156, 7431.25, 7413.06
Duration      [total, attack, wait]             5.012s, 5s, 12.268ms
Latencies     [min, mean, 50, 90, 95, 99, max]  11.292ms, 13.468ms, 13.1ms, 13.838ms, 14.313ms, 24.981ms, 43.191ms
Bytes In      [total, mean]                     705964, 19.00
Bytes Out     [total, mean]                     966056, 26.00
Success       [ratio]                           100.00%
Status Codes  [code:count]                      200:37156
Error Set:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment