I understand your concerns, I should have a look at the style guide. However, how do you feel about the next steps for this proposal? I see we could go two ways:

1. Create a proper PR and fix style and all build targets (windows, osx, all libraries...) in order to merge this release cycle
2. I continue building up in a separate branch towards an initial QML app

Option 1 would mean I don't need to constantly rebase on top of the latest changes, but seeing that it is already quite hard for you to release a new version, maybe it would make matters worse...

Create a proper PR and fix style and all build targets (windows, osx, all libraries...) in order to merge this release cycle

I'm not actively involved these days, so please take my opinion with a grain of salt, but, my thoughts:

• Despite how many files that are touched (446), the refactor isn't outrageous to see merged sooner than later, yet provides an important step towards your goal.
• In my (limited) experience, the UI PRs take forever to merge due to:
  • Getting things pixel perfect
  • Strong (differentiating) opinions over position, placement and design
  • Blockers -- such as artwork, refactors, bloated democratic processes, cross-platform bugs

So, it's my opinion that working towards a separation (of UI and core) -- in steps -- is paramount to making it happen and should be done quickly.

That said, I've been inactive for quite a while now, so my peers' opinion should be weighed over my own.

Hi jsmtux,

I appreciate the feedback you gave about #6472. I listed a few reasons why using Qt in the core may not be a good idea, such as real-time safety. It is a lot of work, I agree, and I also understand that Qt comes with a lot of functionality that may be difficult to replace without bringing in other third-party libraries. In any event, some parts of the code still use Qt versions of functionality provided in the STL, so there should still be an effort to removing Qt as per the guidelines. A STL replacement is like a drop-in replacement, given that we have compiler support for the specific functionality in question. We may not be able to fully remove stuff like QDir and QFile until we get full std::filesystem support for C++17, but classes like QString and QVector should definitely be replaced with their corresponding STL alternative ASAP. My knowledge about QML is rather weak, but the option to use it on the UI side shouldn't be limited by the goal of removing Qt from the core. We will still be using Qt on the UI side, that's perfectly fine, it's just the core where things have to be in real-time and efficient (along with other reasons), which Qt may not fit the criteria for.

There is definitely a lot of opinionated ideas surrounding this topic, so I am also curious to see what you think as well.
Thank you for your contribution, and I am looking forward to seeing a PR by you soon.

Hi sakertooth

You are right, a qt-less core and QML on top can perfectly go hand in hand. My biggest reason for keeping qt in core is indeed not needing extra effort.

The part I would need to learn more about is how performance is currently hindered by qt. Things like containers, parsers or file utilities won't be impacting audio playback. Containers are going to be as performant as STL and file operations need to happen asynchronously in any case.

Audio could be indeed affected by Qt's message and slot handling, since the thread running the event loop gets frequently stopped to do bookkeeping. For sure time sensitive operations like audio processing and so on need to be on their own thread. If that's not the case at the moment, I understand the need of rewriting that specific part of the code without relying in signals, slots and so on.

In any case, thank you all for the interesting discussion! I am very happy to see that my current plans fall in line with your thoughts, even if we don't yet agree on the final goal.

I'll keep you posted with my progress.

Cheers!

so there should still be an effort to removing Qt [as per the guidelines]

What this describes is removing Qt as a build dependency altogether, but what the OP aims to achieve is to separate the UI from the headless components. What makes the OP's efforts particularly attractive (to me at least) is that it encourages the latter and can help with unit tests (e.g. in headless environments). I would even go as far as to argue that the headless functionality is a hard-requirement of this effort, quoting:

I managed to create a core library that is used by a new target, lmms_headless, that produces an executable that can be invoked from command line and has no QWidget dependencies

This can and should be part of the PR's tests.

Qt in the core may not be a good idea

I don't think anyone argues with this point, but removing signals/slots should always be weighed against just fixing what we have (expand the comments, please) since the latter is often what plagues the user experience, and mistakes have been made that they live with today. <3

My opinion is still weighed lowest here, since I'm not as active as others commenting, but my vote is to fix formatting, take @jsmtux's PR and use it as a first step into separation of UI from headless.

My knowledge about QML is rather weak, but the option to use it on the UI side shouldn't be limited by the goal of removing Qt from the core.

I have never been a fan of QML, personally, but what @jsmtux proposes (new UI) would likely start as a fork or as an experimental opt-in feature so that it can be developed alongside a moving codebase and I think a QML proof of concept would go a long way to prove viability.

First of all, thank you very much for this PR. If it can be integrated, it might be a real breakthrough.

The part I would need to learn more about is how performance is currently hindered by qt. Things like containers, parsers or file utilities won't be impacting audio playback. Containers are going to be as performant as STL and file operations need to happen asynchronously in any case.

I once tried to write down the pitfalls with QT's containers. They might look harmless, but often they are lazy-allocated, so it could mean that if the realtime thread does the first ever writing operation it will do a heap allocation (hope it gets clear from the link example).

On the other hand, it might be OK to do the whole thing in 2 steps

• step 1: Separate GUI and core
• step 2: Rewrite core to be realtime safe

Maybe it's even cleaner to separate these two steps?

Thanks for the clarification, Johannes.

On my end, I am focusing on step 1. It will unblock me to try different UI options.

However, step 2 would be also simpler to accomplish after the separation. Having a way of running tests without any ui will hopefully enable better performance analysis and so on.

There's one thing I still don't know if I'll do, which is make all headers private in core and only expose the minimum amount of interfaces to enable the UI. This would simplify making changes in the core, since it will be clear what is used from outside and what can be fully modified. However, it will depend on how long this takes.

Please, let me know your thoughts.

Getting into the matters outlined in your suggestions, the dependency from core to ui is not something I added, or even like, it is just something that was already there.

I didn't mean to imply that you added it, I meant that the usage in core needs to be removed. :)

When compiling against the QWidgets implementation, the code will behave as before did. However, when compiling against Headless, it can be as simple as:

std::cerr << "File not found " << file_path << std::endl

This, I think, fulfills @michaelgregorius requirement of the Core not making any assumption about how actions are actually handled. It is also quite similar to @Veratil suggestion of having callbacks; a pure virtual class is almost the same as a structure with callbacks.

We could even try just using a struct without callbacks, just use std::optional and allow the GUI to handle the failure to load however they want:

/* in Core */
struct CoreConfig {
  type opt1,
  type opt2,
  // etc.
};
std::optional<CoreConfig> core::loadConfig() { ... }

/* in GUI */
if (auto ret = core::loadConfig(); ret.has_value()) { /* config valid */ }

Alternatively, if there are multiple ways to fail loading the config we could use a pair/tuple like:

enum class ConfigError { None, SomeError, ... };
std::pair<ConfigError, std::optional<CoreConfig>> core::loadConfig() {
  CoreConfig config;
  // for error
  return {ConfigError::SomeError, {}}
  // good return
  return {ConfigError::None, config}
}

if (auto ret = core::loadConfig(); ret.first == ConfigError::None) { /* config valid */ }

In any case, I feel that by implementing the proposal, any subsequent changes will be made much easier. Having a folder that shows all calls from core to gui will clearly outline what still needs to be done instead of being hidden from everyone's eyes.

This does help a ton in seeing what connections are still around.

What we have there is a class from Core actually creating a gui Dialog. This is fixed in my proposal, with all dialog creation happening on the ui. This example is, I feel, a much worse offender of what @michaelgregorius was taking about when mentioning that core shouldn't be controlling anything in the gui.

Not a worse offender, but the same level of offense to me. Any connect from core to UI is equally bad.

My hopes are that, even if this is not a perfect change, will be a good starting point to fix both the core and the gui in LMMS.

This definitely is a great starting point, and we appreciate all the time you put into doing this so far. Thank you! 🥇

Alternatively, if there are multiple ways to fail loading the config we could use a pair/tuple

FYI, we can use std::expected once we can use C++23 in LMMS(maybe far future).

Thanks for the conversation! I think we are making good progress by aligning on our differences ahead of the PR creation.

I think the original title was distracting the conversation from my main purpose, which is to enable the creation of a more modern UI for LMMS. This discussion is now called "Proposal for enabling alternative UI frameworks through initial separation of core and UI". If you agree, this will narrow the scope a bit and let us focus in the immediate future.

As you indicated, towards a full separation of UI and Core, my proposal would only go half way. On the other hand you'll probably agree that, for the new objective, both problems (error reporting vs. dialog creation) have different risks and it is worth more focusing on one than the other. As I see it, there are two main problems with the createDialog function:

1. The prototype (virtual gui::ControllerDialog * createDialog( QWidget * _parent );) relies on the existence of QWidgets, making core dependant on the QWidgets library.
2. For createDialog to work, core needs to instantiate the real implementation, which means adding a compile time dependency on the real type. core would need to know if it was instantiating a QWidgetDialog, a QMLDialog or a DoNothingDialog at compile time, instead of leaving that responsability to the UI implementation defined at link time.

Both of those would invalidate the possibility of an alternative framework. On the other hand, just exposing interfaces will allow any alternative (QWidgets, QML or Command line to name a few) to define how they'd like to behave, with the build system only deciding which to use at link time. This means we could compile libLMMSCore.a only once and then reuse that output in the lmms_qwidgets lmms_qml and lmms_headless binaries right away.

The fact that I am limiting this scope will mean that the UI implmentation work will be unblocked faster and that LMMS will be in a better position to integrate your ideas. I feel that if we only aim for a full separation on this initial PR, both objectives will be delayed or even never reach fruition.

Continuing with the example of the saveConfigFile, when the function is called from the UI your solution would work perfectly, core will return an error and let UI deal with it. However, that function is not only exercised by the UI, but also from core itself. In ConfigManager's destructor the saveConfigFile function is also executed. Solving that case would incur in analysing in which moments ConfigManager could be destroyed, find which UI actions start that destruction, and only then rewrite the code accordingly. The same goes, for example for the whole ITimeLineWidget interface, which is a particularily complex case.

Cheers!

@jsmtux, I agree that having the interfaces from Core to GUI will nicely condense where the leftover problems are that need to be tackled to have a clean separation. The goal would then be to replace all methods in the interfaces by implementations that only go from GUI to Core and remove one method after the other from the interfaces.

With regards to Controller::createDialog: from a quick glance over the code it seems that the specific controller implementations are not really used in an abstract or plugin-like way. The Controller class even has the (ugly) enum ControllerTypes which seems to be used quite a lot through the code. I think it might be possible to implement a specific dialog for each Controller subclass so that if the creation of a specific Controller is triggered from the GUI the corresponding controller GUI can be created from the GUI code as well. The dialog will then be given the Controller instance that it controls.

I hope that your code will really get merged as it would improve the code and make it a lot cleaner than it currently is.

@michaelgregorius, thanks for your support.

In the refactoring, I found two different cases for functions inside Core creating GUI elements:

1. All options are known at compile time like Controller or Clip: I created factories that, taking an enum value for the specific type, instantiates the correct instance. ControllerDialogsFactory and ClipViewFactory are examples of this. GUI asks either of these factories when needing a specific controller.
2. Options can be defines at runtime through plugins, like those that inherit from Plugin (Instruments, tools, effects...): I started by removing the gui::PluginView * createView( QWidget * parent ); function for the base class Plugin. That way, Core can use plugins with no extra dependencies. Then, new QWidget specific classes were created inheriting from Instrument, Tool, Effect so that, when needing to call the createView function they cast it like:
   static_cast<QWidgetToolPlugin*>(ToolPlugin::instantiate( desc->name, nullptr )) ->createView(this) );
   Which does work quite well, as any ToolPlugin will be of the QWidgetToolPlugin type when we call it from a source within the QWidgets folder.

So the issues surrounding this should be mostly fixed in my separate_ui branch. It should also build and work for at least linux and macOS, so it would be really appreciated if anyone would like to test it out.

The rest of the work I am doing is under the core_header_separation branch. In that one I am limiting the headers that can be used from the outside of the Core library. This one is, I think, very important to have sorted out before merging anything, but it is still half way done.

Hi @sakertooth , thanks for your interest!

To summarize the progress and my plan a little bit, thelmms_core library is now nicely contained and compiling/linking on its own, but exposes too many of its internals and would be hard to implement an UI to use it. Little by little I am removing files from the include folder, so that GUI implementations only have a reduced, well defined, set of interfaces for all their functionality. It will also mean that a developer working in the core wouldn't impact GUIs as long as the interfaces were kept the same.

In this process, every once in a while I find a block on the road when creating the interfaces. When, for example, there is a bunch of functionality that should be internal from a logical point of view, but because of how it is programmed is hard to make abstract. In those cases I create an extra branch to refactor code a bit, so that I can test changes at smaller increments. One of the blockers was the issue with Model you are referring to, and I used the branch refactor_model_out_2 to make it easier to deal with. That branch should now compile/run with plugins and all just like master does.

I understand the Model stuff might be a bit abstract, hopefully my current task which involves Clips and Tracks will be easier to explain. The main blocker is more or less the same one as before, but the solution I am taking is different. In the case of clips I am not breaking the inheritance as I did between Model and some of its child objects (Composition instead of inheritance), since here the inheritance makes more sense (MidiClip is indeed a Clip).

Changes to Clip structure

In LMMS we have Tracks that contain a vector of Clips. Both of them are inherited by other classes, for example SampleTrack and SampleClip, which are related. In the GUI, a SampleTrackView will hold a pointer to the SampleTrack it contains and will interact with SampleClips.

A naive way of exposing interfaces would be to first create a ISampleTrack, so specific variables and slots like volumeModel or playingChanged in SampleTrack could be accessed. Some functions from Track are also used, like CreateClip, creating the need for an Itrack. And it is here where we find the simplest of this kind problems, the paradigmatic example of Diamond Inheritance Problem as shown in the next diagram:

There would be some ways of dealing with this issue in the simple example above, like Track not inheriting from ITrack. It would force every child class to call manually all the Track functions, but it would be a way out. Unfortunately, some more complex cases do require this link, as I'll explain in a moment. For a while I thought virtual inheritance would get me out of this problem, until I realized QObjects cannot be used this way : ( .

A more convoluted example is the usage of the CreateClip function in Clip. When SampleTrack needs a new clip, it will call that function, which returns a Clip. Of course, being a SampleTrack, the clip it links to will always be a SampleClip that, when CreateClip is called, will be returning an object of SampleClip type. The current code does not explicitly reflect this, so the Clip object is always cast to a SampleClip. Because of it, it wouldn't be possible to just break the inheritance fron IClip to Clip, since CreateClip needs to return a working base interface.

In order to solve this issue I tried to reference the most concrete type I could on each situation. Since we know what kind of Clip will be on each Track, I changed all the places that made reference to the base types to the more specific ones. This had lots of side advantages like no more need for casts all over the place, or giving much more context to whoever is reading the class definition.

With the changes in place, there is no more need to cast from a base class to a child one, making code safer but, more importantly, removing the need for ISampleClip to inherit from IClip. This should work nicely since the GUI layer is quite shallow. Let's say we have a window modifying a PatternClip, it will only be dealing with that kind of Clip, so no need for casting to a different kind. In the case of an elment that needs to show different Clip types, only the common things like names and lenghts will be needed, specifics would be part of another window.

A note on Model refactor

With the context about the changes to Clip, I think it is now simpler to explain previous changes to Model and why it didn't make sense for other classes to inherit from it. In most places where we were inheriting and using Model references like with Plugin, for example, code got much more readable and safe to deal with by referencing the actual child class: a function that accepts Model* and always casts it right away to Plugin* is much less readable than if it just receives Plugin* right away.

There were some classes, of course, where we needed to deal with the more generic part of the Models. An example is when setting the Model's parent. Models are arranged in a tree like structure, so that when a child emits changed parents will forward that change up. Anything using a Model gets notified when any of its children get changed. Up until now, we were just passing this, since the current element did inherit from Model. Now model is just an attribute of the current object, and we can just send the attribute, without the need to expose more.

Thanks

If you read all the way maybe you have some comments or further ideas ; ) . In the following days I'll be uploading the branch I am using for the clip changes, and it will be really helpful if you had a look at it or you testted it. In any case, thanks for the interest.

Alright, looks like I answered from my work account 😅

Wow, very promising demo! A new clean QtQuick-based UI implementation would be awesome. Even though I'm no longer involved in the LMMS project, I've been working with QML/QtQuick in my daily business for many years now and simply love it. Keep up the good work and I'm sure it could lead to a next-gen LMMS 2.

This actually sounds ver similar to the situation I am getting to.
My design would indeed end with a layer that manages sound devices, plugins, tracks songs and little more. Over this core, a layer that, as you say, serves the interface to the qml integration. It sets up the application itself and provides the interfaces from the core in a way qml understands. It would also be dependendant on the os, setting things up a bit differently on Windows or android, for instance. Finally, the gui layer is purely qml, with some ui definitions and javascript interaction.
It is indeed quite a powerful division, since the core part could be used as a pure command line application. In the other hand, changes exclusively to the gui could enable all sorts of different apps only by changing the qml side.
The part where our designs diverge, however, is that in my current implementation the core does contain qt. This is not as bad as it might sound as, since qt5, the framework is very modular and including the qt core module doesn't mean we'd also be including any ui, drawing or OS dependant functionality. You can have a look at the qt modules documentation to see how little is provided by the core module, the only qt dependency we'd have in lmms core. On the other hand it does provide functionality we for sure need there, like message passing which would need to be replaced by something else if we take qt out of the equation.
I hope I'll be able to provide a new build usable from Linux and macos quite soon, that can load/save different Lmms projects and modify dynamically loaded plugin parameters from the gui. With it I will document this layer split I am talking about a bit better and hopefully will allow you or others to investigate this kind of interesting designs, as the core will already be quite self contained.
Thanks for your input!

Just as some food for thought, this sounds vaguely reminiscent of Domain-Driven Design, which I've worked with quite a bit through the ABP framework for web dev, and in that context I can't say I'm the biggest fan. Your design is subtly different, though, and so I think for LMMS it has a lot of potential and is definitely worth considering.

The fork from jsmtux here is not an official branch, so you would still not be able to merge to the official repo. Your best option is to take the time to migrate to a branch off the latest master commit.

For learning about Microtuner, either @LostRobotMusic can give you some directions here or you can join the Discord server and talk with Lost there.

Thanks for the advice.
I was on discord but got impression that it's mostly for the users.
What channel there can I use if I have questions to developers? Or is this chat a good place?

Also, I compiled the latest sources on Fedora 39 and noticed that CPU is always running full speed when I start LMMS with a default (empty) project. The fan kicks in and I don't know how to stop it. I looked through the existing issues and found something similar but it did not seem to be my case.
I don't remember having that issue in earlier versions (last time i used it was 1.13). Any idea as to what the reason could be?
Andrei

Hey! Thanks for the support, always encouraging to get some nice comments!

The PR merging is a bit of a complex topic that I've been giving quite some thought lately. I don't think we could merge my branch as is, it has too much garbage. With all the learnings gathered this last few months, however, there might be a way for getting the work merged in a simpler way.

Current state

The work done up until now consisted of the following steps:

1. Build the core and gui folders into separate compile units
2. Move headers out of the common include folder and closer to the sources
3. Stop gui from accessing the core headers and create a core_interfaces library to provide the access
4. Build a QML gui using the new core_interfaces

The first two need to be done the same way I've done them. They are not too costly in terms of time, and the changes are easy to review (mostly moving files around). The biggest problems were some functions that needed to switch to different files or some global variables that needed some better handling. By far the most time consuming part was the third step, as most of the gui code needed some restructuring to adapt to the new interfaces, this might have taken me 80% of the whole time I've spent in this process. And it doesn't need to be done again.

Merge proposal

The first time around I didn't understand the code as well as I do now and it was impossible to me to know all the functionality gui would need from core. The best way for me to do so was by restricting the header visibility to core_interfaces and fix one by one all the compile errors. That's not needed anymore, as we now know that the functionality now exposed in core_interfaces is all the existing (and any other with the same functionality) gui would need.

So I think work could now be split into two parallel activities:

1. Get the main branch in a state that can support the interfaces from my branch
2. Continue building the QML application on the separate branch

Get the main branch in a state that can support the interfaces from my branch

The first two steps I took need to be replicated, so that the core can be built on its own, producing a static library that gui can use. Once this is done, we could keep the QWidgets gui mostly untouched, continuing to depend on the full core library and all its headers.

Then, a new library would be created core_interfaces, by copying the one are currently provided in my branch. Making core fulfill those interfaces will be quite straightforward, in most cases it will be enough for a class like Song to inherit from ISong, and that's it. For things like AutomatableModel or Clip things will be a bit messier, but not too much.

This would be a great moment for documenting the interfaces, making them more consistent and removing things from them that we think won't be needed by the new gui.

Continue building the QML application on the separate branch

If you were to compile my qt_6_qml_poc branch for Linux or MacOS you'd see that there is quite some functionality already working, with support for a few plugins and some interactivity. But it is far from complete. I think this is quite valuable as the more fleshed out this alternative is, the more help we'll get for getting it mainlined. Also, the more we'll understand about the interfaces and what we want from them.

One of the tasks that I think could be a good starting point for other developers interested in QML development could be porting some of the plugins so they can be used here. At the moment I only have kicker, audiofileprocessor and tripleoscillator working, and only the last of those with a useful QML interface. Once I manage to update the code to allow users to browse through their song libraries I'll also try to write a little bit on how to approach this development. Making another plugin available shouldn't be too much work, and will be a good starting point for helping with other elements in the application.

It might sound inefficient to continue working on this separate branch, but once the previous step is finished it will be trivial to port all this work to mainline. This depends on the core_interfaces, so as long as those are mostly the same everything will continue to work.

What do you think?

Quite soon I'll post another update to the prototype that hopefully will prove the idea is worth it, supporting quite some features and looking decent. But from then on I'll need support driving this forward, as it is not possible for me to put everything in place and much less keep up with the development in master.

So let me know if this plan for the merging makes any sense to you. Once I push the next update I'll try to be present in the Discord to support you building or modifying the code.

Thanks for the super detailed reply!

I am totally on board with getting steps 1 and 2 done and PR'ed ASAP, and it seems like you've basically done the job already.

The next steps also seem good, although (absolutely nothing personal) I'm not as keen on some of the specifics of those, and think they need more discussion - especially among core maintainers smarter than me - before we actually go ahead with them.
For instance, I'd be more in favour of using C++ 20 modules to control the visibility of core interfaces, rather than the proposed core_interfaces layer. I'm not trying to criticize anything you're doing at all, and I'm by no means a project expert; I just think it's worth pausing for formal discussion following the completion of steps 1 and 2.

And let me know if you need any help with it, because I am happy to lend a hand! I am on the LMMS discord server, or just technoporg for DMs.

Sadly it is really not recommended to mix both. I understand the hesitation about a big bang approach, but the fact that the whole gui needs to change at once wouldn't mean all changes need to happen at once.
A possible alternative could be target LMMS 1.4 to be widgets based but also providing a proper interface as previously described. LMMS 1.5 to improve on such interface but keeping existing widgets. And finally 2.0 building the new guide on top of the new interfaces and removing the old one.
It is just an example, but I am sure together we can find other better alternatives.

Great job @jsmtux. You might want to look into the qt6 branch where we got working qt 6 builds and we seem to get stock LMMS working on Qt6. If you don't mind, I would advise you to wait a bit longer for that branch to get merged to master and then work on top of the new master. Once that's done, you can just enable the "WANT_QT6" cmake flag and work on top of that. Best of luck.

Thanks for your reply!
We'll, this branch is just a prototype, wouldnt want to merge as is. What I'd like to merge are the changes for supporting the core/gui separation, things like compiling them into separate objects or providing interfaces.
I expect that to take a few months anyway, by that time I am sure the qt6 branch you mention will already be in place.

Hi LMMS team,

Maybe you've already had a chance to look at my PR #7341 . I've been keeping it up to date with the latest master and moved some elements in the src/gui/include folder into src/gui, as suggested by some reviewers.
I've also spent some time in preparing what could be the next step, where I do some minor changes to remove dependencies from core to TrackView classes. This might be interesting so you can see the kind of changes that would go into removing all dependencies from core to gui. As you see it is a bit painful, but not too dangerous. You can have a look at it in this PR https://github.com/jsmtux/lmms/pull/3/files .