Which programming language did you choose and why?

[Arbiter]

Processing power is growing - but RAM size grows much faster than CPU single-threaded performance.

At the same time CPU speed is increasing much faster than RAM throughput so cache friendly code and data structures are more important than ever.

 

[Gambler]

Unity killed many ambitious games (like Hellion) - not because less than optimal framerate - but because of this inherent nature of GC - and random pauses every few minutes.

There is no such thing as "inherent nature of GC". Different VMs and languages have different garbage collector implementations with drastically different performance characteristics, settings, runtime controls and so on. Moreover, it is an active area of research and performance in most languages changes over time.

For example, I don't know much about Unity, but judging by a recent news release, they still use Mono, which is severely lagging behind .NET in terms of GC quality. Mono 5 introduced incremental GC as a feature preview at the end of 2018. Hellion was released for early access in 2017. What GC was Hellion using?

Also, different languages have different strategies for dealing with CG pauses. Java has many GC-related settings you can tune. .NET/C# favors optimizing code by using various memory-related keywords and structs. Go developers decided to specifically optimize GC to reduce latency at the expense of everything else. Etc.

In short, making overly generic statements about GC is paramount to lying.

 

[PorkyThePaladin]

You just keep getting more and more upset, just like in the Elden Ring thread, or the Shitmaker thread before that. Like this other guy:

I give up. I swear I tried, but you're so entrenched in this cringy "trolololol" persona that it's impossible to have a reasonable conversation with you.

Just please take a step back and realize that I came here saying "I love Python" and your immediate reaction was "you're still butthurt, you're obsessed with me, cry some more". I don't know what being always so on the defensive means, but seeing you act like this every single time is kinda sad.

You know, you can't lie this blatantly when everything is recorded in this thread:

I love Python and how easy it is to do stuff with it even for inexperienced people, but my last experience with it went something like:

"Boohoo, our beautiful program is perfect but it runs for hours what can we do"

"Here, I rewrote everything in C++"

"NO WHAT ARE YOU DOING THIS IS HORRIBLE... wait, why does it run for just a couple of seconds now?"

Granted, it was a very fringe case of a custom, complex algorithm to solve unreasonably large linear problems and one can't draw general conclusions from it, but it's still painful to have to deal with self-made programmers who think Python is literally our lord and savior.

So basically: I luv Pythonz BUT it's slow and it sucks, and I will post for the first time after you do, cause you live rent free in my tiny head...

Yeah, ok, bud. If you can't stand the pork heat, get out of the kitchen...

 

[PorkyThePaladin]

Python performance issues are completely overblown here. It is a relatively slow language compared to others, but what these people won't tell you (for you young programmers out there):

1. For many real world applications, performance doesn't matter that much. For example, any kind of scripts, automation, tools, scientific programs, etc generally are not real-time, meaning if they have to run for some time, it's not a big deal. If you are running an automated script for some office process for example, it doesn't matter if it runs for 3 minutes or 15 minutes in most cases, because it will be running in the background.

2. Modern hardware is extremely powerful, with CPUs being able to do something on the order of billions of operations per second. Meaning a benchmark test that shows that C++ is 40 times as fast as Python is meaningless in most real world applications, because a C++ program will finish in let's say 0.01 seconds, and a Python program finishing 40 times later, will finish in 0.4 seconds. Do you think anyone will notice the difference in the real world?

3. Most real world slowness in programs comes not from the programming language but from other sources: IO, DB connections, Internet, or badly written code. These things will generally dwarf whatever performance differences there are between languages.

The stuff above might not apply to everything, I wouldn't use Python for truly fast code, e.g. 3D games or operating systems or those financial programs that need to submit transactions in fractions of a second or a website that serves tens of millions of hits daily, but for the vast majority of real world applications, Python speed is not a big issue.

 

[Rincewind]

Nowadays - I would say that even predictable 500% overhead would be perfectly acceptable - but if GC needs to defragment allocations of old generation objects - it will stop all threads of application for as long as it needs. If you give it more memory - you sometimes get exponentially longer pauses - just less often.

And 'professional software' is not that much different. I've seen deployments where GC was disabled and instances were restarted one by one after midnight to free memory - because it was more predictable than letting GC do its job.

What you wrote regarding stop-the-world GCs is certainly true, and that's representative of how the GC in the JVM works.

However, it doesn't have to be like that. There are GC implementations that don't "stop the world" and have been specifically designed with hard real-time (!) requirements in mind. With real-time GCs you can have full manual control over the collection frequency, plus you can specify the max allowed collection time when you trigger a collection cycle manually. Moreover, such GCs don't need to be global either; they can be local per thread.

There's another school of thought that with full manual of reference counted memory management freeing memory can also introduce long pauses when you're freeing huge object graphs, for example. Whereas with a manually controlled GC you can just chip away at it in small chunks at periodic intervals, e.g. at the end of every frame in a game.

Nim offers a great deal of flexibility when it comes to memory management. You can do full manual management, ref counted, or you can switch between several GC implementations with different strengths and weaknesses (e.g. Go, Boehm, MarkAndSweep). This post summarises the various memory management options in Nim nicely, just quoting some of the more interesting parts:

Nim's GC is also thread local, so each thread has an independent garbage collection. This means one thread's collection doesn't halt all other threads as with the Boehm GCs (which you can select from Nim if you want this). Additionally it means you can create threads in Nim that do not use GC types and thus do not use GC at all. "Stop the world" collection aka halting all threads to collect garbage is a problem in many other GC languages due to sharing the GC across threads, and really hits performance for example with multithreading in Python with it's GIL.

In real terms, Nim's GC is almost never the problem with performance. If you want to see what the GC is up to, you can use GC_getStatistics. You will see zero time is being spent even if you're hammering it, which is hard to do. In my largest project, which is performance sensitive, I get about 2,000 scans every 5 seconds. That's nothing. Much bigger impact is from actual work that needs doing and how cache friendly something is, which is down to program design.

You can also use GC_Strategy to declare if you want throughput, memory usage or responsiveness for example. You can use the realtime GC and force it's collection period within milliseconds, you can even tell the GC where roots are, what structures are acyclic with the {.acyclic.} pragma (useful for some types of trees) and even step the GC manually or force collections when you want.

 

[Rincewind]

The stuff above might not apply to everything, I wouldn't use Python for truly fast code, e.g. 3D games or operating systems or those financial programs that need to submit transactions in fractions of a second or a website that serves tens of millions of hits daily, but for the vast majority of real world applications, Python speed is not a big issue.

That's a level-headed post of yours, and if you started out with something like this, I think there would have been a lot less drama. Personally, I still don't like the forced imperative model, mutation everywhere, and lack of basic functional features in Python, but just to get the job done in scripts and automation Python is fine, can't disagree with that. I would also be the first to admit that imperative programming is a lot simpler to pick up for people who only sporadically write a few scripts but aren't full-blown coders (e.g. many scientists, data analysts, etc). And I like the syntax in general, and one of my favourite languages, Nim, is heavily inspired by the Python syntax.

I think the problems started when you went into retardo territory and started claiming that strongly typed languages are *only* for idiots and cheap outsourced teams...

Anyway, I'm willing to put that behind us

 

[PorkyThePaladin]

The stuff above might not apply to everything, I wouldn't use Python for truly fast code, e.g. 3D games or operating systems or those financial programs that need to submit transactions in fractions of a second or a website that serves tens of millions of hits daily, but for the vast majority of real world applications, Python speed is not a big issue.

That's a level-headed post of yours, and if you started out with something like this, I think there would have been a lot less drama. Personally, I still don't like the forced imperative model, mutation everywhere, and lack of basic functional features in Python, but just to get the job done in scripts and automation Python is fine, can't disagree with that. I would also be the first to admit that imperative programming is a lot simpler to pick up for people who only sporadically write a few scripts but aren't full-blown coders (e.g. many scientists, data analysts, etc). And I like the syntax in general, and one of my favourite languages, Nim, is heavily inspired by the Python syntax.

I think the problems started when you went into retardo territory and started claiming that strongly typed languages are *only* for idiots and cheap outsourced teams...

Anyway, I'm willing to put that behind us

I did start with this, you were just too butthurt to notice because you started raging when I criticized your preferred languages. And you are still twisting my words, Python is fine for websites and any kind of programs really, short of a realtively small niche of high performance applications.

And I never said strongly typed languages were for idiots, I said they were more idiot-proof and thus a better fit for larger teams with varying quality of developers. Do you really not see the difference in what I am saying and what you are saying?

 

[Rincewind]

you were just too butthurt to notice because you started raging when I criticized your preferred languages

You're talking about yourself here now, right?

Python is fine for websites and any kind of programs really, short of a realtively small niche of high performance applications.

For example like all mobile apps and all embedded devices? Hello? Or anywhere where you need to ship a *binary*, not just some unencrypted, free-to-read-for-the-whole-world script and a 40-50 megs runtime for your 1k script? The world is not just automation scripts and websites...

And I never said strongly typed languages were for idiots, I said they were more idiot-proof and thus a better fit for larger teams with varying quality of developers. Do you really not see the difference in what I am saying and what you are saying?

Giving it right back at ya: "And you are still twisting my words" (And conveniently ignoring or side-stepping facts we have presented when it doesn't align with or undermine your "agenda"...)

Anyway, a lot of knowledgeable people have shared their views on the merits of various programming languages and paradigms in this thread; the information is there if you're willing to step outside of *your* niche of Python development. Have a nice day, fella.

 

[PorkyThePaladin]

you were just too butthurt to notice because you started raging when I criticized your preferred languages

You're talking about yourself here now, right?

Nope, you.

Python is fine for websites and any kind of programs really, short of a realtively small niche of high performance applications.

For example like all mobile apps and all embedded devices? Hello? Or anywhere where you need to ship a *binary*, not just some unencrypted, free-to-read-for-the-whole-world script and a 40-50 megs runtime for your 1k script? The world is not just automation scripts and websites...

You keep trying to use this bad sophistry, but it's like 2nd grade level. Nobody is saying that Python or any other language for that matter is a great fit for every situation. My point, which you might get one day, is that performance wise, it can fit most real world scenarios. But in other ways, it might not be best fit for a particular task, just like C or Kotlin or Scala or whatever.

And I never said strongly typed languages were for idiots, I said they were more idiot-proof and thus a better fit for larger teams with varying quality of developers. Do you really not see the difference in what I am saying and what you are saying?

Giving it right back at ya: "And you are still twisting my words" (And conveniently ignoring or side-stepping facts we have presented when it doesn't align with or undermine your "agenda"...)

Lol, those were literally your words I quoted, unlike you, who completely changed mine. Just admit you were wrong and deal with it, bre.

Anyway, a lot of knowledgeable people have shared their views on the merits of various programming languages and paradigms in this thread; the information is there if you're willing to step outside of *your* niche of Python development. Have a nice day, fella.

It's not my niche, I worked with Java, C, Ruby, Assembly, other stuff, I am just here to defend a great language against silly people like you.

 

[Arbiter]

I am just here to defend a great language against silly people like you

No, you are here to defend Python against Guido van Rossum, whom you called a Python hater.

 

[Arbiter]

I am just here to defend a great language against silly people like you

No, you are here to defend Python against Guido van Rossum, whom you called a Python hater.

What's next, Bill Gates the Windows hater secretly coding Linux patches in his mansion?

 

[Azdul]

Unity killed many ambitious games (like Hellion) - not because less than optimal framerate - but because of this inherent nature of GC - and random pauses every few minutes.

There is no such thing as "inherent nature of GC". Different VMs and languages have different garbage collector implementations with drastically different performance characteristics, settings, runtime controls and so on. Moreover, it is an active area of research and performance in most languages changes over time.
(...)
In short, making overly generic statements about GC is paramount to lying.

Modern GCs change the way VM deals with new generation of objects - those that survive shorter than few seconds without deep web of interdependencies or are accessed from single thread.

In case of Hellion more modern .NET GC would not have any impact.

There are only three strategies of compactifying the storage of old generation objects:
- 'stop the world' for 'as long as you need' - used in every GC available from OpenJDK, Oracle, Microsoft or IBM
- restart application
- pointer indirection on hypervisor / ring 0 level, duplicating every single allocation, dedicating one core exclusively to GC, predictable global 20 ms pause every second with limitation on heap size - available from Azul Systems

Yes - you need to fundamentally change how hypervisor and operating system work with memory to have GC that will never 'stop the world' for longer than 20 ms. And GCs were supposed to make programmer's life easier

 

[Arbiter]

Mono 5 introduced incremental GC as a feature preview at the end of 2018. Hellion was released for early access in 2017. What GC was Hellion using?

Hellion lived until 2020, plenty of time to upgrade to Mono 5.

 

[Burning Bridges]

Unity killed many ambitious games (like Hellion) - not because less than optimal framerate - but because of this inherent nature of GC - and random pauses every few minutes.

Yes Hellion great example. I really loved that game. It was so exciting to collect stuff from space wrecks, fill containers with vital material like Helium and Oxygen and put them in neat shelves. And another day come back to your storage and see half the contents had dissapeared from the containers.

I would really be interested how this multiplayer bug came about when position of objects was not in sync and ships suddenly jumped several hundred meters, sometimes causing collisions. Was that related to a lack of understanding of floating point variables? or an attempt to "optimize" that caused the most bizarre of bugs? in any event it was this that mostly killed the game because the idea of precision space docking was sabotaged by ships jumping around you like monkeys on a stick

The cherry on top was that the servers were rebooting what felt like every 60 minutes, because the GC and whatever else they had done caused it to come to a stop.

To sum up: Hellion was working fine, until more than a few users started logging in an playing.

Kerbal Space program had such issues too, but they sugarcoated or ironed them out, by an inhumane amount of work I guess. The Hellion guys on the other hand just gave up because the game did not sell well, because it was so fucking hard (and awesome).

 

[Krice]

In short, making overly generic statements about GC is paramount to lying.

GC is a "solution" to a problem that doesn't really exist. If you get memory management problems in non-gc languages you simply suck as a programmer. It's simple. GC was designed for modern languages and "modern" programmers and I don't want to point out who they are. We all know.

 

[Krice]

2. Modern hardware is extremely powerful, with CPUs being able to do something on the order of billions of operations per second. Meaning a benchmark test that shows that C++ is 40 times as fast as Python is meaningless in most real world applications

This has been proven wrong so many times I don't know why people keep repeating this lie. You are talking to actual programmers here, no need to convince anyone. You can make programs slow really fast if you don't know what you are doing. Python is slow, no one has proven anything else yet. Show us the fucking programs written in Python that run fast, or shut the fuck up.

 

[Burning Bridges]

What bothers me as a programmer is not wether an algorithm takes 5 minutes instead of 10.

What matters are millions of micro interruptions because apparently every keypress takes seconds to execute. Every second waiting for a keypress is lifetime and amounts to gigantic productivity and motivation loss. I want kepyresses to be almost instantaneous, only after that can be guaranteed we can talk about choosing programming languages.

A great example is the new Microsoft Flight Simulator where every action in the main menu takes up to several seconds, because they apparently bulk-imported 20 year old code as html and parse it ever time, or must generate 100 thumbnails from a 9MB file to 9kb because someone forgot to make requirements to save the thumbnail as a separate file.

I would say you can write absolutely ok code in Python or any language, but if you can't make sure that you are not executing lots of inefficient, redundant middleware through bottlenecked interfaces, the whole user experience can be pure hell.

I have for example seen performance increases not of 400% but 100,000% by switching from a "convenient" COM interface to a TextStream and plain txt. Language didnt matter in that case because C# was used.

And this will happen a lot in those convenient, very high-level languages where the programmer only has to write a single line, and doesn't know wtf actually executes it.

 

[Azdul]

Unity killed many ambitious games (like Hellion) - not because less than optimal framerate - but because of this inherent nature of GC - and random pauses every few minutes.

I would really be interested how this multiplayer bug came about when position of objects was not in sync and ships suddenly jumped several hundred meters, sometimes causing collisions. Was that related to a lack of understanding of floating point variables? or an attempt to "optimize" that caused the most bizarre of bugs? in any event it was this that mostly killed the game because the idea of precision space docking was sabotaged by ships jumping around you like monkeys on a stick

I believe it was GC stopping the world every few seconds once new player got connected. Since the QuakeWorld every reasonable programmer knows that if you don't get new packet from the client - client is moving in the same direction with the same speed. On the other hand - there are plenty of crazy new game companies who think that experienced programmers will accept 2000$ a month ...

No, we don't - enjoy your startup crash and burn.

 

[Tramboi]

Python performance issues are completely overblown here. It is a relatively slow language compared to others, but what these people won't tell you (for you young programmers out there):

1. For many real world applications, performance doesn't matter that much. For example, any kind of scripts, automation, tools, scientific programs, etc generally are not real-time, meaning if they have to run for some time, it's not a big deal. If you are running an automated script for some office process for example, it doesn't matter if it runs for 3 minutes or 15 minutes in most cases, because it will be running in the background.

And for many it does. And when it doesn't you can still use the other languages.
Quite often, something (another program)/somebody is waiting for the result of your automated script somewhere, and those 12 minutes mean the world.
I think saying that non-real-time programs don't have latency needs is a fallacy

2. Modern hardware is extremely powerful, with CPUs being able to do something on the order of billions of operations per second. Meaning a benchmark test that shows that C++ is 40 times as fast as Python is meaningless in most real world applications, because a C++ program will finish in let's say 0.01 seconds, and a Python program finishing 40 times later, will finish in 0.4 seconds. Do you think anyone will notice the difference in the real world?

Or you're working with computers and you can do 40 times more computation-bound work in your work day.

3. Most real world slowness in programs comes not from the programming language but from other sources: IO, DB connections, Internet, or badly written code. These things will generally dwarf whatever performance differences there are between languages.

Agree about the "badly written code" part.

The stuff above might not apply to everything, I wouldn't use Python for truly fast code, e.g. 3D games or operating systems or those financial programs that need to submit transactions in fractions of a second or a website that serves tens of millions of hits daily, but for the vast majority of real world applications, Python speed is not a big issue.

I think you're downplaying it. IMO, Python is unsufferably slow and you have lots of high level languages that are more expressive AND much faster.

 

[Tramboi]

In short, making overly generic statements about GC is paramount to lying.

GC is a "solution" to a problem that doesn't really exist. If you get memory management problems in non-gc languages you simply suck as a programmer. It's simple. GC was designed for modern languages and "modern" programmers and I don't want to point out who they are. We all know.

No GC is very old and was in McCarthy's Lisp.
GC has its uses in high level languages, it enables very convenient abstractions, we just don't want it in realtime programming (yet ?) nor systems programming (ever ?).

 

[Burning Bridges]

Since the QuakeWorld every reasonable programmer knows that if you don't get new packet from the client - client is moving in the same direction with the same speed.

I don't know, what if it isn't and the packet has just not arrived yet?

I thought the problem was that the same variables existed many times. Everyone had positions on their computer but they were already outdated, or in the worst case just estimated by some "clever" method (i.e. a hack). Then suddenly the real value arrived from the server and it was completely different. Something was completely wrong here, maybe the updates that should have happened several times per second were just not coming and since they were only using someone else's libraries they never figured out how?

It's also possible that they were dumb enough to program their server in Unity too, because that was all they knew. The clients running in Unity was probably never an issue. But the server had to be able to support ultra high response and precision, ensuring that no matter what the important position synch was guaranteed to be happening within milliseconds, and no bullshit like garbage collectors or random Unity bloat able to interfere.

They should have solved this before they embarked on a real time multiplayer online game, doing it without understanding the outcome was incredibly dumb and naive, which seems to be the trademark of mediocre Unity programmers.

 

[Burning Bridges]

No GC is very old and was in McCarthy's Lisp.

GC has its uses in high level languages, it enables very convenient abstractions, we just don't want it in realtime programming (yet ?) nor systems programming (ever ?).

Of course, it is not that using a gc language is making you a worse programmer. The lack of memory management results in less clutter in the code, and you save time that you can use for other things. But if a gc makes the application stop and make unpredictable pauses, getting worse after longer runtime, this disqualifies it for a lot of purposes. For business applications this is almost never a problem, because people are staring at a 2D UI where not much is happening anyway. But most games cannot afford to stop randomly at least if they have multiplayer.

 

[Lutte]

The stuff above might not apply to everything, I wouldn't use Python for truly fast code, e.g. 3D games or operating systems or those financial programs that need to submit transactions in fractions of a second or a website that serves tens of millions of hits daily, but for the vast majority of real world applications, Python speed is not a big issue.

These are far from the only type of software where the performance of larger scripts will hit severe bottlenecks in real world usage. A industry veteran, Mitch Kapor, once tried to write an outlook clone (mail-agenda organization type software) in python, called chandler.. it was a s-p-e-c-t-a-c-u-l-a-r failure entirely imputable to the runtime just not cut for handling massive amount of data in real time, no one wants to use a piece of software whose UI is unresponsive.

 

[Burning Bridges]

I've also seen a complete CAD system written in Java by some dork from a university. Same problem, the constant interruptions and waiting times drove you nuts.

 

[Viata]

It's mostly a few utilities that modders have written for various games.

From my perspective it's games. If you can't create a game with some language it's pretty useless. Why would I use it then, to do what?

But you can create a game with any language. Graphical games? It depends if the language has any graphics library, but if it's a text-based, yes, you can do it with any language.