Chainer has a "define by run" architecture for defining computation graphs, so you can have very dynamic computation graphs. TF/Theano have static computation graphs, though an optimized define by run library is coming to TF in ~February, not sure if there are any plans to make it play nice with Keras though.
Ohh that's very similar to ours actually. Our computation graphs just work relative to their context. We've found it very easy to add arbitary vertices because of it. Not that you care, but someone might: We ended up using this style because we opted for not having auto diff in our layers due to speed as well as the state of our framework at the time (layers being hand implemented already).
We got the use case for auto diff but not being focused on research we just decided it was easier to hand implement the layers and just have the graphs be defined at the layer level.
