You can install a separate desktop environment such as Xfce - but that would limit you to apps that uses the X window system protocol. I.e. that would be apps that you are probably familiar with from systems such as Linux. This usually also means that you loose all of the macOS platform specific features of such apps. I.e. it's not really a viable choice for most.

Another path to take would be to try to configure or otherwise modify the macOS desktop environement to your likings. There exists a huge number of third party applications that change stuff like how window management works, add keyboard shortcuts, etc.

Also there's the way of simply getting used how things work on macOS before changing everything. For example your impression that the maximize button "doesn't work" probably comes from the fact that you're used to other windowing systems like Windows where the maximize button is used to change the size of the window to fit the desktop - on macOS this button is used to change the size of the window to fit the contents of that window. That is a much harder task than just fitting the desktop, and some application developers do not provide the necessary support for this feature (by declaring the size of window contents) - and therefore you can have edge cases, where this doesn't work as expected.

Note that macOS also has the "maximize to fit desktop" feature. On a standard setup, you can get that by holding down the Option key and pressing the green plus button in the top, left part of a window.