Some rethinking about framework

Feature Suggestions
1

Hi, team guys. Working with foreign code I have a lot of painful. Not only because of the lack of documentation, for me it was never a problem if your code was written intelligently. Fuse is great framework in the ideological sense and I believe you put your heart and soul into this project, but unfortunately the implementations in some cases lame.

  • You have a lot of redundant and not understandable parts especially in Graphics. To cite one example.

    You have got a lot of structures for matrix operations such as: float4x4, Matrix and FastMatrix… FastMatrix? Is it affine matrix (4x3) or what? I have a lot of experience in computer graphics and image processing but never heard about “fast matrices”, I know affine matrix but it hasn’t sense because it have very little overhead on cpu and gpu and nothing overhead on vector processors at the same time has a terrible implementation and overhead conversion mat4 <-> mat4x3, by the way GL ES 2.0/WebGL 2.0 don’t support it at all, only mat2, mat3 and mat4. So, some methods in your graphic elements can return float4x4 some of them FastMatrix. It’s just a dump of some sort!
    float4x4 has “operator +”, “operator -”, mul and div by scalar, but operators rotation aka Matrix * Vector and concatenation Matrix * Matrix are missing. Why? It looks like just as container. So we need use static methods from Vector.Transform and Matrix.Mul. It’s looks terrible and understandable! Look at Matrix in Unity for example. The same problem with vectors.

  • For user drawing stuff especially for drawing elements with texture I need actual width and height corrected for any type of StretchMode. You haven’t any public methods or classes for that. Besides, I needed cropped UV-coordinates for that. You have internally private float2 _origin, _scale, _drawOrigin, _drawSize, _uvClip for that, but user don’t access to them, so I need reimplement of this. In forum I found SizingContainer class in bolav’s thread, but it contains bugs in some cases it works incorrectly!

  • You havn’t got in Uno very basic build-in type/class in many languages, I mean RegExp. Why you create you own Runtime instead using Runtime from Mono? Your language Uno is superset of C#/Mono right?

  1. You draw strokes inset instead of perimeter centre for geometry. This produce artifacts in rotated or skewed geometry, so trick to masking aliased edges by draw antialised stroke over it not working in default case! I resolve this by tuning Stroke.Offsetbut if I decide animate stroke’s width I should animate offset as well!

  • If I transform Image I expect that it will be applied to the texture coordinates and not to the geometry as a whole. In other words, I can’t compensate for distortion of the image after parent’s transformation at all. I’m working on reimplement this behaviour in custom element but it’s really hard.

  • Import from Sketch not working since 0.23.

  • Textures look creepy without mip-maps

  • When I use Element.CaptureRegion for rendering to framebufferIt seems you use multisampling by default on. It’s not good idea. I would like control this, because multisampling is very expensive. And another issue with framebuffer is premultiply alpha, because elements drawing by self in screen and rendering to framebuffer look quite different and have darker edges.

This is not all of problems, unfortunately I don’t have time to describe everything. Maybe I’ll do it later. Fuse is a great product, but in my opinion it now needs stabilization and doc’s improvements instead of growing functionality. Since you have a small team and if you want to attract community to help to make your product better, it may be a good idea to open part of the code dealing with the basic abstractions in Fuse namespace.

Thank you for reading this text till the end)

2

Hi Max,

Thanks for your feedback, Max. It is very exciting to see users dig this deep. As you can tell, our main focus is to polish the UX/JS user experience. The Uno layer is what we use internally to implement all of this, and unfortunately is not as well documented, yet. It was however designed for a lot of requirements that might not be that apparent off the bat.

I’m not denying there is some technical debt in the code base, but all in all we are quite happy with our framework design. The team behind Fuse is very tech savy, with all together hundreds of years of experience in graphics programming. The Fuse code is optimized over a long time for a variety of use cases.

You have got a lot of structures for matrix operations such as: float4x4, Matrix and FastMatrix… FastMatrix?

FastMatix is a class used internally in our Visual heirarchy which optimizes the common cases where a matrix only contains a simple 2D translation (VERY common case in flat 2D UI). It tracks whether a matrix is only a flat 2D translation and avoids a the full matrix multiplication in that case, while still allowing full 3D transform when neccessary. This optimization gave us a lot of speed-up when it was introuced.

Is it affine matrix (4x3) or what? I have a lot of experience in computer graphics and image processing but never heard about “fast matrices”, I know affine matrix but it hasn’t sense because it have very little overhead on cpu and gpu and nothing overhead on vector processors at the same time has a terrible implementation and overhead conversion mat4 ↔ mat4x3, by the way GL ES 2.0/WebGL 2.0 don’t support it at all, only mat2, mat3 and mat4. So, some methods in your graphic elements can return float4x4 some of them FastMatrix.

We are fully aware of all the quirks and limitations of GPUs and GL, and can assure you the outmost care has been taken to ensure performance in the cases that matter.

It’s just a dump of some sort! float4x4 has “operator +”, “operator -”, mul and div by scalar, but operators rotation aka Matrix * Vector and concatenation Matrix * Matrix are missing. Why? It looks like just as container. So we need use static methods from Vector.Transform and Matrix.Mul. It’s looks terrible and understandable!

No, this is very logical, and carefully thought out.

Just like a float can represent many things, such as degrees, dollars or distance, we don’t add a bunch of methods to float with these assumptions, do we?

Uno is a hybrid GPU/CPU language. The floatN and floatNxN types are built-in types known by the compiler, exchangeable between the processors. This means the types must be completely general purpose.

Therefore the floatN and floatNxN types in Uno are semantic-free, as in they have no idea what interpretation they have. For example a float4 could be a Color, a Quaternion or a Vector. The operators found on floatN are only the “pure” ones where there is no ambiguity. For the rest, you have to use static methods based on how you interpret the vector.

To simplify your code, you can do:

using Uno.Vector;

And then

float3 a = ..;
float3 b = ...
var c = Dot(a, b);

Looks neither terrible nor incomprehensible to me.

For user drawing stuff especially for drawing elements with texture I need actual width and height corrected for any type of StretchMode. You haven’t any public methods or classes for that. Besides, I needed cropped UV-coordinates for that. You have internally private float2 _origin, _scale, _drawOrigin, _drawSize, _uvClip for that, but user don’t access to them, so I need reimplement of this. In forum I found SizingContainer class in bolav’s thread, but it contains bugs in some cases it works incorrectly!

Intersting, please report this bug separately with code to illustrate your problem.

You havn’t got in Uno very basic build-in type/class in many languages, I mean RegExp. Why you create you own Runtime instead using Runtime from Mono? Your language Uno is superset of C#/Mono right?

No, Uno is a subset of C# that compiles into pure C++ with no dependency on the large, slow and bulky mono runtime, nor any dependency on the .NET framework. This gives smaller, faster apps that in the future can also be exported to copmact web code/ASM.js.

You draw strokes inset instead of perimeter centre for geometry. This produce artifacts in rotated or skewed geometry, so trick to masking aliased edges by draw antialised stroke over it not working in default case! I resolve this by tuning Stroke.Offsetbut if I decide animate stroke’s width I should animate offset as well!

Edges of our shapes are anti-aliased by default, without relying on hardware antialiasing. This shouldn’t be a problem.

If I transform Image I expect that it will be applied to the texture coordinates and not to the geometry as a whole. In other words, I can’t compensate for distortion of the image after parent’s transformation at all. I’m working on reimplement this behaviour in custom element but it’s really hard.

I totally disagree that it should be applied to the texture coordinates - that makes no sense. All other elements rotate the entire element when you rotate it. Why should Image be different?

Perhaps what you mean is a separate feature that allows you to rotate the image coordinates, that’s a good feature suggestion that would be great to have in a separate thread.

Import from Sketch not working since 0.23.

There is this: Fuse

Textures look creepy without mip-maps

Couldn’t agree more! However, vanilla GLES2.0 doesn’t support mipmaps on non-power-of-two textures.

Our plan here is to let Image implement it’s own super-pretty downsampler instead. It’s on our todo-list, just haven’t gotten around to it yet.

When I use Element.CaptureRegion for rendering to framebufferIt seems you use multisampling by default on. It’s not good idea.

It doesn’t, as vanilla GLES2.0 doesn’t support multisampled framebuffers.

What you are seeing is our Fuse’s magic shape shaders antialiasing without relying on the hardware antialising. Nifty, huh? :slight_smile:

I would like control this, because multisampling is very expensive.

Don’t worry, it’s always off.

Fuse is a great product, but in my opinion it now needs stabilization and doc’s improvements instead of growing functionality.

We totally agree, so that’s the plan!

Since you have a small team and if you want to attract community to help to make your product better, it may be a good idea to open part of the code dealing with the basic abstractions in Fuse namespace.

Yes - open source is on the way! Just a lot of work to do it right.

Thanks a lot for your input, looking forward to hear more. Talk to us on Slack ? :slight_smile:

3

You draw strokes inset instead of perimeter centre for geometry. This produce artifacts in rotated or skewed geometry, so trick to masking aliased edges by draw antialised stroke over it not working in default case! I resolve this by tuning Stroke.Offset but if I decide animate stroke’s width I should animate offset as well!

I think for the stroking issue, you’re looking for the Stroke.Alignment property. We defaults to Inside, since this gives sharp result when anti-aliasing strokes on pixel-snapped coordinates, the same way as e.g Photoshop does. You can change this to Center to get the behavior you seem to want. Using center will let you animate the width without having to adjust Stroke.Offset.

4

And another issue with framebuffer is premultiply alpha, because elements drawing by self in screen and rendering to framebuffer look quite different and have darker edges.

Yeah, we’re aware of this. This isn’t an issue by itself, just something you need to be aware of when doing custom Uno code.

I have a plan to fix this inconsistency, by making all images use pre-multiplied alpha (to remove some edge-artifacts), but I haven’t gotten around to this yet. Until then, you’ll need to either un-premultiply the colors that comes from Element.CaptureRegion, or pre-multiply colors that doesn’t. When the switch comes, we’ll make sure this is clearly visible in the changelog.

5

For user drawing stuff especially for drawing elements with texture I need actual width and height corrected for any type of StretchMode. You haven’t any public methods or classes for that.

Are you sure? We have Element.ActualSize, which I believe should be exactly what you’re looking for.

6

Thanks for speed feedback.

FastMatix is a class used internally in our Visual heirarchy

Ok,so make it really internally and hide for users. FastMatrix is part of Fuse namespace, not Fuse.Internal and Visual can operate with it in methods called LocalTransformInternal, PrependImplicitTransform etc. Take into account the disadvantages of the docs all that making confuse.

Therefore the floatN and floatNxN types in Uno are semantic-free, as in they have no idea what interpretation they have. For example a float4 could be a Color, a Quaternion or a Vector

So, that is just features of the architecture. I understand this, but as customer I prefer use something like this

float d = (mat * p.Normalized()).Dot(t.Normalized());

Instead of this

float d = Vector.Dot(Matrix.Transform(Vector.Normalize(p)), Vector.Normalize(t))

First is much clearer, isn’t? And what if I need change order of multiplication for Matrix and Vector? In first case I just swap two arguments like this: p.Normalized() * mat and that’s very easy!

To simplify your code, you can do:

using Uno.Vector;

And then

float3 a = …;
float3 b = …
var c = Dot(a, b);


Is just make my code little shorter but not clearer and flexible.

> Edges of our shapes are anti-aliased by default, without relying on hardware antialiasing. This shouldn't be a problem.

For geometry its true, but not for an ```<Image>`` element:
``` 

<Image File="Assets/whiteRect.png" Width="400" Height="400" Color="#fff">
    <Shear DegreesY="3"/>
</Image>

I totally disagree that it should be applied to the texture coordinates - that makes no sense. All other elements rotate the entire element when you rotate it. Why should Image be different?

Ok, Maybe transform nodes like <TexCoordinatesMove/>, <TexCoordinatesScale/>, <TexCoordinatesRotation> and etc is better idea but this functionality really lacking!

Couldn’t agree more! However, vanilla GLES2.0 doesn’t support mipmaps on non-power-of-two textures

Yes, we already argue about this and I hope you’ll give some tools for resolve this during compile or runtime process.

It doesn’t, as vanilla GLES2.0 doesn’t support multisampled framebuffers.

Vanilla 2.0 not, but wide range devices support it by extensions like GL_APPLE_framebuffer_multisample on iOS and GL_EXT_framebuffer_multisample on Androids, and of course ES GL 3.0.

Maybe it’s just some issue described to the above about <Image> aliasing.

7

Are you sure? We have Element.ActualSize, which I believe should be exactly what you’re looking for.

Element.ActualSize an Element.ActualPosition not working for Image with different kind of Stretching. Even LocalTransform and WorldTransform unusable when element has complicated padding/margin and contained in ScrollPanel. I will give an example a little later