Fiber, X, Ad Blocking and Tracking

It’s been about a month since my last update? Maybe I should post something.

Fiber has been a bit slow moving, partly because I’ve just been slow and hitting other projects, but mostly from toying around with CEF. One prominent issue is the fact that the current crop of CEF examples and samples are using a mix of GTK and X. Many outdated questions and forums on the web bring up Qt and CEF, but with the relatively limited time I have to work on the project it has made things feel slightly more arduous than it needs to be when there’s no answer to the problem. Part of this is the fact that Qt has WebEngine/WebView, so CEF integration is a niche topic. It’s nothing insurmountable, but I might need to accept a temporary dependency on X. This annoys me, but the project needs to move forward and I can’t be stuck spinning my wheels. At the lowest level of the application I will need platform-specific code anyway, I simply wanted to see how long I could avoid it.

“Did you ever solve for X?”

(more…)

The Fiber Engine Poll, Updates, and Breeze

Some weeks ago I ran a poll to see what would be the preferred rendering engine for Fiber, and so I figure now is the time to post results. There was a surprising amount of misinformation/confusion running around about what each option potentially meant which I hope to clear up, but overall the results were so compelling I doubt stripping the misinformation and re-running the poll would return a different result.

Third Place: Port to CEF Later

“Porting to CEF later” was the lowest voted option at ~18% of the ballet, and in retrospect it makes sense since it’s just a poor solution. The only upside is that it gets an obsolete implementation out the door (if that’s an upside), but it makes things complicated during an important phase of the project by putting an engine change in motion while trying to flesh out deeply tied APIs. Not good.

Oddly some people wanted a WebEngine/CEF switch and took to this option as Fiber having such a switch. Considering CEF proper is based on Chromium/Blink (which is what WebEngine uses) it’s a bit like asking to take two paths to the same destination; there are differences in the road but in the end both ways lead to Blink. There will be no switch for Cef/WebEngine because adding one would bring down the API potential to the lowest common denominator while increasing complexity to the most advanced method.

Runner up: Use WebEngine

“Use WebEngine” was the runner-up at 24% of the vote. The main prospect behind this is that it would result in a shipping browser fastest, but it also works under the assumption that it may increase code compatibility between Qt-based browsers – but the architecture of Fiber I believe will be very alien compared to contemporary solutions. If there are chances to collaborate I will, but I don’t know how much of that will be possible.

There was also a segment that voted for WebEngine thinking CEFs was just a more complicated route to Chromium, being confused about the road to Servo.

Winner by a mile: Go Exclusively CEF

It’s no surprise that in the end “Use CEF” trounced the remainder of the poll at 59% of respondents voting in favour of it – more than both other options combined or any individual option doubled. From the comments around the internet one of the biggest reasons for the vote is Servo as a major differentiating factor between other browsers, and also because it would help mitigate the Webkit/Blink monopoly forming on non-mozilla browsers for Linux.

This excites me as a web developer, and I’m likely to try pushing Servo as the default engine as it will likely be plenty good by the time Fiber is released. Sadly, I believe there were a few votes placed thinking that Fiber would ultimately usher in a “QCef” or “KCef” framework; and I don’t think this will be the case.

On making a Frameworks 5 API I considered it as a super-interesting Frameworks addition, but after careful consideration I realised there just aren’t too many projects which would benefit from what would be a substantial amount of work. Another issue is that I think the QWebEngine is appropriate for most projects, and that anything more is needless complication. The Qt developers have done a good job picking the right APIs to expose which suits common needs, and I imagine the additional complexity would only hurt projects adopting such a library; it’s killing a mosquito with a cannon. Plus, QWebEngine will evolve in good time to fill any common needs that pop up.

What will Fiber do?

Fiber is going to go exclusively CEF. I’m in the process of fiddling CEF into the browser – but CEF is a bit of a beast and about 3/4 of my time is simply reading CEF documentation, examples, and reading the source code of open projects using the utility. My main concern is properly including CEF without requiring X11; it’s possible, but the Linux example code isn’t using Aura, and the implementation examples are GTK-based as well. Qt and KF5 have solutions, but I’m reseaching the best route to take.

In terms of what engine Fiber is using (Servo vs Blink) I’m going the generic route; you can drop in simple config files pointing to CEF-compatible executables, and when configuring profiles you can pick which engine you would like to use based on those files. This engine switch is already present on the command line and in the “Tuning” section of the profiles manager. This means you can have different profiles running different engines if you choose. There’s a second command-line option which will launch a new instance of Fiber with the engine of your choice running one-time for testing purposes. For the purposes of the default, I’ll probably push Servo.

CEF will not drive UI

Indirectly using CEF means QML may become the exclusive language of UI extensions, popups, and config dialogs. Mainly this is because of the additional abstraction and effort required to offer CEF in several contexts, but it also puts a much cleaner separation between browser and content and will likely make securing the system easier. Extensions will be required to offer pages in HTML.

If you’re using QML, your writing chrome. If you’re using HTML you’re writing a page.

This is also more in-line with the Plasma Mobile guidelines, and though I severely doubt you’ll see Fiber become a mobile browser any time soon this keeps the door open for the far future. In two years I’d rather not break a significant number of extensions for mobile inclusion; I’d rather just have things work, maybe with some minor layout tweaks.

There are real pros and cons to QML as the only way to extend the browser UI, and probably one of the largest I worry about is the fact that QML has a significantly smaller developer base than HTML. On the plus side QML is able adapt to platforms, meaning we might not need to divide extensions between desktop and mobile – that would simply boil down to layout tweaks. All this means is instead of having many extensions of questionable quality, we will aim to offer fewer but higher-quality extensions.

On Progress

Progress is steady. Probably an hour to two of work a night goes into the project, and extra time on weekends as freedom allows. It drives people nuts that I’m taking my dear sweet time on this, but when the groundwork is done there will be a solid base for others to help quickly build on.

I’ve introduced threading into some parts of Fibers management tools, and made significant improvements with how Fiber manages internal data caching for profile data. This all got started when I noticed a split-second of lag on a slider, and realised the long-term implications. Threading was introduced so when the database models are working they do not lag the main thread, and the layer which talks to the model now caches the data and only communicates with the model when one is out of sync. The next step will be to add some internal very coarse timers and event tools which will delay hard data saves until they can be batched efficiently or must be written, and possibly a check to prevent the saving of idenitcal data.

While this may not matter as much for the management tools I’ll be applying these techniques on an extension-wide bases; this will save power, keep Fiber highly responsive, make it CPU wake friendly, and avoid hard drives wakeups – even when bad extensions might behave in “thrashing” behaviours. Ironically this first performance exercise has made me confident that even with many “slow” javascript-driven features, Fiber may become a highly performant browser by virtue of having extremely fine-tuned APIs which give blanket improvements.

One of the most annoying but necessary changes was porting Fiber from QMake to CMake. Originally I had the intention to prototype using QMake, switching to CMake later for the “real” work. As things would have it the prototype had simply evolved and I realised it would just be easier to port it. As I’m not terribly familiar with CMake this started off painfully, but once I realised what CMake was trying to encourage I fell in love and things just clicked.

During the CMake port I also took the opportunity to strip out vestigial or prototypical code and do some housekeeping, which certainly cleaned things up as I not only removed files but also disposed of bits of code too. I also removed all traces of WebEngine which I had used during the earliest prototype phase; the next time Google pops up, it’ll be with CEF.

I’ve also started incorporating the first KF5 libraries into the project. The libraries are very well organised, and also well documented. Finally, I need to compliment Qt and state how amazing the toolkit is. Really. Some of the most notable changes were trivial by Qt making smart use of its internal structure, and even though I’m hardly a veteran developer Qt and it’s extremely good documentation has allowed me to make smart, informed decisions. Really guys, good job.

On other projects

Moving away from Fiber, right now we’re doing a lot of work on refining the Breeze theme for Plasma 5.5 in this thread, where we’re running down paper-cuts on the design and building the next iteration of the style. Ideally, we’d like to see a much more consistent and well-defined visual structure. Later on we will start to address things like alignment issues, and start targeted papercut topics which will address specific visual issues. If you’re interested, please read the entire thread as there is lots of design discussion and contribute your thoughts.

Remember, constructive feedback is the easiest contribution anyone can make to an open-source project!

Fiber Update; WebEngine vs CEF

Fiber has seen some active development, and over the course of a long 3-day weekend full of hacking I’m glad to say that exactly 0 progress has been made! Of course that would be a bit of a fib, I’ve spent the weekend re-factoring all of the profiles code and organising the codebase structure.

I also spent a good chunk of my time reading Qt and KDE coding guidelines and documentation on how files and classes should be structured, and then I applied that information to Fiber. The result now is well commented code, and consistent naming conventions in-line with other Qt/KDE projects.

But re-factoring code isn’t what I’m really interested in talking about…

WebEngine vs CEF

When I started Fiber I worked under the assumption that WebEngine would be the engine for this browser; it’s an official Qt extension, being actively developed, and isn’t going anywhere. After Fiber kind of came into the light I had a comments and emails pointing me to CEF, the “Chromium Embedded Framework” as an alternative to WebEngine.

After doing research it’s severely divided my thoughts on what to use.

What is it?

CEF started as a Chromium-based project meant to create a stable API relative to the rapidly changing engine, something non-qt applications could use as easily and reliably as Qt applications do with WebView . While it started off as just an implementation CEF has a defined stable enough API that it turned into a sort of pseudo-standard. Servo, Mozillas new wonder engine is actually building itself to be a native CEF implementation, meaning that future Firefox will actually be a CEF-based browser.

CEF, despite being not so well-known, is actually used by some very high-profile companies which lends credence to the longevity of the project. Adobe, Valve, Blizzard, Amazon, and other big names have thrown their chips behind CEF. Valve in-particular bases their Steam client on the thing.

Pros and Cons

Cons

Not everything is rosy and bright in the world of CEF; there are always downsides. The first and biggest downside is the fact that CEF doesn’t have a Qt library. The Qt guys didn’t decide on this arbitrarily as they have a different goal for the WebEngine API. At minimum CEF means more complicated work than using an established Qt API.

CEF and having multiple engine options also means that we may see two entirely different sets of bugs coming in, depending on whether or not a person is running Fiber-Chromium or Fiber-Servo in the future. This doesn’t even include potential future CEF implementations; who knows what might show up in 5 years.

I would also like Fiber to be extremely portable, which makes CEF more of a concern; WebEngine currently supports mobile, but CEF will only have Android support ‘in the future’. Since Plasma Mobile includes a more malleable stack I have less doubts that Fiber will run fine on that, but I would like to see Fiber eventually run on Android.

Finally, CEF will add a lot of weight to the browser as an external dependency, to the tune of at least 40MB at minimum. This is more due to WebEngine being part of Qt and already being on the system – but CEF isn’t, and so the rendering engine is a separate binary to distribute. If a distro ever eyes Fiber as a default browser it means there’s over 40 extra reasons to consider a slimmer browser which makes use of more common libraries. Granted, just about every major browser is packing pretty big binaries anyway – but it’s still wasted space.

Pros

One thing that’s kind-of well-known is that WebEngine doesn’t have a particularly deep API (yet). For most applications this is fine as the utility is just used to display some content and hook a few things in so the app can talk to the page in an efficient manner. Fine for the use-case that Qt envisions. For a full-on browser though WebEngine lacks a lot of important interaction points, and though I Initially doubt Fiber will be able to make use of deeper integration in a meaningful way, as time goes on it’s a serious advantage to lose, especially since I don’t know the roadmap for WebEngine and what they plan to add.

WebEngine and WebView also have bad security reputations – I don’t know the specific issues, I just know those issues are prominent enough to see Fedora choose not to ship it. CEF doesn’t seem to have this perception as far as I know. That being said I’m not a guru-level C++ programmer so I’m not disillusioned to the fact that I’ll inject my own security shortcomings; though I won’t have the worry of breaking downstream applications in the quest to fix those issues.

There’s concern in the web development community of a WebKit/Blink monoculture. Outside of Gecko, there’s no longer any rendering engine variety for the Linux community. While I doubt Blink will ever “slack off”, the fact is Blink has undue weight over the web because of its sheer dominance. With more variety it means Blink has to keep in-line with the wider browser community rather than driving it. Gecko, Trident, Edge, and Servo all push Webkit/Blink harder in the same way many desktop environments push KDE to be better.

But the absolute biggest advantage in my opinion is the fact that CEF will offer Servo in the future as well as Blink. It means that we will be highly portable in our choice of rendering engines, able to swap out quickly. If we tightly integrate with Webengine it means Fiber won’t have mobility in the future, but through CEF if one of the engines gains a significant technical lead we can change the engine easily.

The Poll

We have three options for Fibers engine of choice, and I’d like the people who may ultimately use the browser to decide, as I’m really truly torn!

  1. Stick with WebEngine. It’s simple, easy, fast, supported. Fiber already has WebEngine running.
  2. Start with WebEngine and just get the browser up-and-running, later make the transition to CEF. It would be fast at first, but it will complicate things later on as we make the transition, especially if there’s several extension APIs connected with WebEngine.
  3. Write Fiber to use CEF natively. This will probably result in a more performant browser, and will allow deep integration; though it would still be slower to develop than just using WebEngine.

[polldaddy poll=9012714]

(Don’t see the poll? Click here!)

Finally, if you have comments, please post them below and elaborate on what drove your decision.

KDE.org Redesign

KDE Frameworks, Plasma desktop, and our community have a rich history of nearly twenty years in creating great open-source software, making us a truly historic organisation of passionate developers; and along with that history some of our online infrastructure has begun to show its age. The KDE.org website and its various sections are the front door to the KDE ecosystem, it is how people new to KDE will judge us and it’s where our developers, translators, artists, and community members know their hard work will be presented to the world.

Recently there’s been a minor movement in the KDE community hive-mind stirring about our web network, and it’s increasing need for a significant update. Some of this has been separate, some in groups, some know about the others but not vice-versa. There was a BOF at Akademy which also focused on a new website and goals.

We’ve opened up a new community forum with the dedicated task of modernising our legacy infrastructure to create a unified effort. Everyone wanting to participate will have the goal of revitalising the KDE.org website for our 20th anniversary, giving us a strong infrastructure to celebrate a stronger foundation for the next 20 years of KDE, community, and open-source.

The WWW team, Promo Team, and VDG have all expressed desire to build a better KDE.org, and we’d all like to see the wider KDE community to contribute! You don’t need to be a web developer or server administrator to chip in; even if it’s a few minutes of feedback or a single idea, we would like to see you participate! We plan to build this website on a foundation of respect and openness – everybody and all constructive input will be welcomed.

There’s no specific goal to have the whole network completed; there’s a huge number of assets and each one is unique, but I believe it’s reasonable to have many of our most prominent pages and sections converted in time for our 20th anniversary – October of next year, roughly. Beyond that, we’ll work section-by-section page-by-page to ensure KDE.org is as amazing as it can be.

Click here to go to the forum