How to run REAL python scripts from go

Showcase: running real python scripts from go apps

In this showcase, we’ll show the journey and the technical concerns we had to allow end-users to write custom python code (including pip packages) from our go app.

Spoiler alert: hard core experiments ahead

Brief abstract:

1. Why the heck do you need to do that? let’s talk about our use-case
2. What are the technical concerns? well, running in production for (near) real-time use cases isn’t easy…
3. What have we tried and failed to do? wasm, go-python-binding, compile python to starlark and bind to go
4. What we built? sidecar architecture: Python grpc server that communicates with a go grpc server over unix-sockets + AST to loosely restrict the operations
5. What’s next? how can we improve this architecture

Bouns: this is OSS!, if you find these kinds of challenges interesting - come and contribute some code :P

Credit to Miki for ideas and brainstorming around the challenges