Created
February 16, 2015 10:24
-
-
Save peterbourgon/4d817dc1acc08e469052 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
<ptrb> mattheath: curious if my idea of the interaction between | |
Transport and Codec maps to yours... viz. | |
http://play.golang.org/p/BgcgWvkSSR | |
<mattheath> yeah think so | |
<mattheath> i’d see the codec as the serialisation part between the | |
transport (http/zmq/rmq) and the server itself which handles a decoded | |
request | |
<mattheath> there are some pretty blurry lines though | |
<mattheath> previous implementations i’ve worked on we used rabbitmq | |
and protobuf by default, but the underlying serialisation part supported | |
both json and protobuf and put a header on the wire indicating which the | |
payload was | |
<ptrb> so one concrete codec implementation with a sort of type | |
switch? | |
<mattheath> i guess yeah, it wasn’t very abstract | |
<mattheath> switched on a content type header (actually a rabbitmq | |
header) | |
<mattheath> hrmm rather irritating that github hides inline comments | |
when i modified something nearby but not the actual thing ;) | |
<ptrb> :\ | |
<ptrb> if all server endpoints will implement the same interface, | |
then I guess the tricky part is figuring out how to transform a service | |
interface method to that form. `go generate`? package reflect? | |
<ptrb> that is, I'd expect maybe something like `type MyService | |
interface { Process(int) error }` --transform--> `func (x | |
MyServiceProcessAdapter) Serve(ProcessRequest) (ProcessResponse, error)` | |
<ptrb> so Request and Response would I guess have to be interface | |
types | |
<ptrb> or did you have something else in mind? | |
ptrb & # afk a little while, but will read shortly | |
<mattheath> ah ok, so I’m used to having a standardised Handle | |
interface - something like Handle(server.Request) (server.Response, error) | |
rather than the finagle style of function as a service | |
<mattheath> and then dealing with that in the handler with any custom | |
logic | |
<mattheath> that then also simplifies the server as it takes a | |
request, identifies the correct handler, and executes it with the same | |
interface each time | |
<mattheath> though appreciate it makes the handlers more verbose | |
<mattheath> as you have to unmarshal your request body inside the | |
handler due to the type system | |
<Vendan> I'm actually working over a tool that'll generate wrappers | |
around functions, stuff to turn a function like func Add(x int, y int) int | |
{} into a pair of structs for easy codec handling | |
<mattheath> what would that output though? | |
<Vendan> my current goal is to have it be a codec generator, tell it | |
an service and it'll generate out structs and helper functions to call the | |
service's functions | |
<Vendan> with the end goal being a function that you can pass a | |
message into and it'll call the service, then return the response message | |
<Vendan> initially going for a json codec generator, but I'd like to | |
make it generic | |
<mattheath> ok, interesting. sounds a bit like thrift? | |
<mattheath> i’ve used protobuf a lot, which gives you structs and | |
serialisation. but usually had a defined client interface to call another | |
service - maybe something like: | |
<mattheath> client.Call(serviceName string, request, response | |
proto.Message) error | |
<Vendan> well, I want to make it a little simpler to consume | |
<mattheath> simpler? underneath it marshals to the on the wire | |
format, sends the request, and puts the response into the struct | |
<mattheath> or gives you an error ofc | |
<mattheath> also you have service discovery, deadlines etc in that, | |
so you probably need a configurable client | |
<Vendan> not sure it'll really fit into gokit, but I'd like to spit | |
out "client side" packages that are the same interface as the service, and | |
it handles the stuffing the structs with the params and stuff | |
<mattheath> so like http://golang.org/pkg/net/rpc/ ? | |
<mattheath> but generated? :) | |
<Vendan> yeah, but no | |
<Vendan> lol | |
<mattheath> ha | |
<Vendan> generated code, and no fixed 2 struct, one in, one out type | |
stuff | |
<Vendan> so you could do z, err := client.Add(x, y) | |
<Vendan> and it'd do all the magic to get it across the wire and | |
everything | |
<mattheath> not sure there’s much difference tbh? | |
<mattheath> just moving around params | |
<mattheath> err := client.Add(struct{int, int}{x, y}, z) | |
<mattheath> uglier i admit ;) | |
<mattheath> but the data formats i usually use are a lot more complex | |
than adding numbers | |
<mattheath> appreciate its just an example | |
<Vendan> lol, yeah | |
<Vendan> part of it is getting rid of reflection | |
<Vendan> and ensuring types match and everything | |
<mattheath> any particular reason for getting rid of reflection? its | |
used in a lot of places anyway | |
<ptrb> mattheath: what I'd *really like* to avoid is requiring gokit | |
services to implement their own Handle function to bridge the Go domain | |
(type MyService interface) to the, for lack of a better term, SOA domain | |
<mattheath> tbh i’ve written services that do 10’s of thousands of | |
requests per second with sub-ms response times and casually ignored the fact | |
my requests and responses were using reflection for marshalling internally | |
<ptrb> but I haven't thought about the problem long enough to know if | |
it's infeasible, especially with all the touchpoints | |
<mattheath> (sorry dumb q) by Handle function you mean? actually | |
setting up a server and handling requests? | |
<ptrb> I mean as you put it: " a standardised Handle interface - | |
something like Handle(server.Request) (server.Response, error)" -- which I | |
presume you'd expect all gokit servers to implement | |
<mattheath> think we have a difference in terminology - i see servers | |
having many handlers | |
<mattheath> unlike (from what I can tell) finagle where a function is | |
a server? please correct me if i’m wrong | |
<ptrb> I don't know enough about Finagle to speak on that | |
<mattheath> ah k | |
<Vendan> one of my personal goals is to be able to make a generic | |
golang library into a service with minimal modification | |
<mattheath> Vendan, interesting, that’s definitely not my use case ;) | |
<ptrb> I'm thinking right now that a gokit server would map one-to- | |
one with a type MyService interface { ... } | |
<mattheath> ptrb: so my thoughts on the handlers were, simple server | |
library - i register some handlers with a name and a func to execute, and | |
they do their thing | |
<ptrb> and maybe you have one Handler per MyService method | |
<mattheath> i’ll knock up something quick | |
<ptrb> but I'd want those handlers to be created by the gokit | |
server.New(myService, ...) | |
<Vendan> ptrb, would a service interface have multiple functions? | |
<ptrb> Vendan: yes, it would be in the business domain of my | |
organization | |
<Vendan> also, is there any particular reason interface and not | |
struct? | |
<ptrb> type LikeService interface { Like(UserID, TweetID) error } | |
<ptrb> Vendan: yes, of course; services are defined by their behavior | |
(interface) not their implementation (struct) | |
<ptrb> Vendan: more concretely, I should be able to pass a | |
MockLikeService to server.New() and it would work exactly the same | |
<Vendan> wouldn't you still pass LikeService, cause that's the | |
interface?but somewhere else the server decides to create a MockLikeService? | |
<ptrb> Vendan: yes, you'd still pass a LikeService, but it can be | |
backed (implemented) by anything: a real implementation, a mock | |
implementation, etc. | |
<Vendan> k | |
<ptrb> var s LikeService ; s = NewRealLikeService(...) /* or | |
NewMockLikeService() */ ; server.New(s) | |
<mattheath> http://play.golang.org/p/ul_CJ_xilk | |
ptrb reads | |
<mattheath> but with interfaces naturally ;) | |
<Vendan> hrm, does the server even understand the difference at that | |
point?I mean, unless you are generating a lot of code, that server.Now is | |
just looking for an interface{} | |
<ptrb> Vendan: the server would only care about the interface | |
definition, so it can generate the appropriate stubs, invoke the Go-domain | |
methods, and perform the appropriate decoding/encoding | |
<ptrb> mattheath: in this example, RegisterEndpoint takes Endpoints | |
that implement (or wrap implementations of) my business logic? | |
<Vendan> is server.New in this example generated code though?are you | |
talking about generating code on the fly? | |
<mattheath> ptrb: yeah RegisterEndpoint would take something that | |
satisfied a Handle() interface that contained your business logic, and the | |
server would deal with getting requests to and from your code | |
<ptrb> Vendan: no, server.New is not generated code. and I'm not sure | |
at what stage the stubs would be generated, it could be at compile time | |
(e.g. `go generate`) or initial runtime (e.g. `package reflect`) | |
<Vendan> also, that endpoint type def is off, it would need a field | |
named Handle in the struct | |
<Vendan> but that's minor | |
<mattheath> Vendan: there are a lot of things wrong with the code i | |
wrote ;) | |
<mattheath> sorry! | |
<ptrb> mattheath: right. if at all possible, I'd like to implement my | |
business logic in code that had no concept of server.Request or | |
server.Response | |
<mattheath> ok, without a standardised input and output, how would | |
the types work? (unrelated, the request would likely be, or implement a | |
context interface) | |
<ptrb> mattheath: and I'd like the mapping of my business logic | |
functions to server.{Request,Response} to be done for me, though I'm willing | |
to cede that point and force users to write mappers for a MVP | |
<ptrb> mattheath: either through generated stubs, somehow, or | |
reflection on the business logic interface, somehow | |
<mattheath> definitely possible | |
<Vendan> heh, that's the first stage of what I'm working on, ptrb | |
<ptrb> Vendan: excited to see your first draft :) | |
<Vendan> well, haven't had much time to work on it yet | |
<mattheath> FYI this is rather similar to chuhnk’s implementation: | |
https://github.com/asim/go-micro#init-server | |
<Vendan> weekends are busy for me | |
<Vendan> but I'll probably have a PoC out tomorrow | |
<Vendan> I do contract work, so I can set aside a hour or 2 to work | |
on it | |
<ptrb> mattheath: I see, yes | |
<ptrb> mattheath: but -- this was good. I feel like we understand | |
each other :) | |
<mattheath> tidyed up the typos in the code here: | |
http://play.golang.org/p/FV7gLMpMJa | |
<mattheath> night :) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment