Manual merge of fix point and linear/new-stuff

[jeffreyrosenbluth]

Not Ready to Merge

This type checks and includes all of the work @byorgey did
on fixpoint. This should replace the fixpoint branch but please
don't delete fixpoint yet as it may have valuable comments that I
missed.

Next

• Contextualize Trace.
• Add Transformable instance for RTree.
• Possible Transformable instance for QDiagrams.
• What to do about Measure in the Context.

[jeffreyrosenbluth]

@byorgey
What happened to the three transforms in the Context from the original manifesto? Did you just not get to it yet or did you think of an alternative way of handling it. Based on the current implementation of meaure, I think something like:

type Context v n = Style v n
                     ::: Name
                     ::: SubMap v n
                     ::: (n, n)
                     ::: ()

might do it.

[jeffreyrosenbluth]

I'm not sure how this definition can make sense after we add Measure n to the Context? This diagram will not be a function of the Global to Output or Normalized to Output transforms.

[jeffreyrosenbluth]

Hopefully this is a reasonable start. I haven't looked at it in a while and there are plenty of holes.
It basically follows Brent's original fixpoint branch to its logical conclusion and incorporates the changes
made to 1.3.
At least it compiles! @byorgey perhaps you can have a look and we can discuss the next steps?

[cchalmers]

I’ve just had a look at this branch. I didn’t realise it was so invasive. It looks like you won’t be able to get any information from a diagram without it being wrapped in Contextual. diagrams is already a pretty difficult library to get started with and I feel that this would make it even harder. For a feature that, although pretty cool, probably won’t get used most of the time.

It looks like this is actually very similar to #246 except for Name and Style not being part of the Context, and a measured diagram isn’t wrapped in a newtype. Isn’t it possible to get this working without rewriting/breaking everything?

It seems to me this could be done by offering a different version of functions for working with any monad, Contextual/Measured/OnlineTex/State etc. This would mean exiting code won't break / get more complicated and make it easier for working with diagrams wrapped in any monad.

[jeffreyrosenbluth]

I basically agree with you about it being much more invasive than I think we originally imagined. Moreover, the original thought was that there wouldn't be many changes to diagrams-lib, but in fact I think it would turn out to be quite invasive there as well, I started on a fixpoint2 branch of lib a while back, but haven't looked at it since.

On the other hand, I think it would be used very often. If you go back and read @byorgey 's fixed point manifesto (https://groups.google.com/forum/#!topic/diagrams-discuss/kBz3xewBe_A) , you willl see it would be used all over the place, everywhere from giving arrows an envelope to taking line width into account when calculating a diagrams envelope.

It differs from #246 in a more fundamental way, it changes the semantics of what a diagram is and allows for as many iterations as necessary to reach convergence. Quoting the manifesto "Under this new framework, delayed leaves can be seen as a sort of hacky way to move things from the 0th iteration into the 1st iteration;"

I'm still not sure the added complexity is worth it, maybe there is a less invasive way to achieve our goal?

[cchalmers]

You're right, it's not like diagrams/diagrams-lib#246. I meant Context ≅ Measured in that it's a Reader over the measures (Context just includes more). Combining / getting the width etc. are very similar.

I also have performance concerns. I've already been looking at ways of improving performance because making complicated plots can get slow but this looks like things would get even slower.

I hadn't read the manifesto before, but I've given it a couple of reads and there's certainly lots of interesting features there (although I still don't understand transforms get applied). I'd just wish it was an "optional extra" rather than being forced.

[byorgey]

In principle, I like the idea of it being an "optional extra". It might be worth thinking hard about whether that would be possible. You can "escape" into the monadic Context world, do whatever you want, and then run a fixpoint to get back into the normal, non-Context world.

[jeffreyrosenbluth]

Having reviewed @byorgey's "manifesto" I thought I would summarize and comment on the things we lose and gain.

Things we could eliminate

• Delayed Leaves - I don't feel that the elimination of delayed leaves is that much of a benefit.
• splitFills - Yes it would be nice to have a cleaner implementation here.
• coproducts - no big deal.
• DualTree and DTree - Dual tree is dramatically simplified and DTree eliminated in @cchalmers simplify-dual branch.

Things we gain

• Arrow envelopes - This is not as big a deal as I thought since the current envelope includes everything but the width of the head and tail.
• Stroke width in envelope - This would be nice.
• Constraint based layout - ?
• Clean up withName and friends - There is probably another way to handle this.

My conclusion, after a peek at what the code starts to look like after fix point is implemented, is that it is not worth the complexity. I would be happy if we could find an alternative way to give envelopes to arrows and stroke width, and possibly explore alternatives to constraint based layout.

thoughts?

[cchalmers]

Some thoughts:

Split fllls

I don't think any backends currently need split fills. As I understand split fills would be useful if we applied styles like this:

<g fill="rgb(0,0,0)">
  << lots of shapes here >>
</g>

but most (all?) backends apply fills on a per path basis so it's easy to check (I do this in the pgf backend). The problem is not all backends combine styles the same way diagrams does (which led to diagrams/diagrams-svg#66). I guess we need to decide if the slightly more concise output is worth all that complexity.

The current split fills would also need to be rewritten to work with simplfy-dual.

Coproducts

I don't think we're using the full potential of coproducts / down annotations. Currently transforms aren't O(1) but they could be. If I applied rotateBy (1/4) 20 times to a diagram, everything in the diagrams would have to rotation function applied to it 20. But if used matrices for transforms, the only the transform matrix (stored in the down annotation) would be multiplied and when the diagram gets compiles each point only needs to be multiplied by one matrix, giving us O(1) transforms.

I've written most of the individual bits to do this, I just need to bring it all together. Hopefully this will give a reasonable performance bump.

dual tree

Glad you mentioned simply-dual (dual-tree and core diffs), I almost forgot about it. It's not quite finished but there's not much left to do (if we ditch split fills). It gives a modist performance bump from not having to go through RTree and DTree (around 10% IIRC) (I think matrix transforms would be a bigger boost), I still need to play around with strictness and things.

I also started sub diagram traversals based on that branch. I managed to get it working but wasn't sure on the details (I tried to give an explanation here).

Stroke width in envelope

We can currently do this if we limit it to local stroke widths. Maybe we could some helpers for this.

withName and friends

The way I've got sub diagrams traversals working requires a rewrite of how names work internally. It could give an opportunity to add any extra features we want.

[byorgey]

At this point I am also leaning in the direction of thinking that going to all-out fixpoint semantics would be much more trouble than it is worth. I like @cchalmers' ideas above, but unless I am misunderstanding, the point is that we can do much/all of these improvements without switching to a fixpoint semantics, right?

Right now, for me the biggest benefit of fixpoint would be the stuff with names, but I am pretty sure we can get that as a separate thing on the side.

[cchalmers]

Right, all the performance improvements and traversals are for the dual tree approach (without fix point).

[jeffreyrosenbluth]

It seems like we are in agreement. So as a plan of action I propose:

• Finish simplify-dual.
• remove split fills, while we are at it we should remove StateStack like we did in SVG and Rasterific.
• Work on making transforms O(1)
• Explore the withName design space.

Also there seems to be a few areas where using local measures has benefits, e.g text, arrow and stroke width envelopes. We should take advantage of this and perhaps provide the user better tools to make use of it.