Over the past one and a half years, Stack Overflow has lost around 50% of its traffic. This decline is similarly reflected in site usage, with approximately a 50% decrease in the number of questions and answers, as well as the number of votes these posts receive.

The charts below show the usage represented by a moving average of 49 days.


What happened?

    • bionicjoey@lemmy.ca
      link
      fedilink
      arrow-up
      47
      arrow-down
      1
      ·
      1 year ago

      ChatGPT has no knowledge of the answers it gives. It is simply a text completion algorithm. It is fundamentally the same as the thing above your phone keyboard that suggests words as you type, just with much more training data.

        • bionicjoey@lemmy.ca
          link
          fedilink
          arrow-up
          26
          arrow-down
          1
          ·
          1 year ago

          Yeah it gives you the answers you ask it to give you. It doesn’t matter if they are true or not, only if they look like the thing you’re looking for.

            • thingsiplay@kbin.social
              link
              fedilink
              arrow-up
              19
              arrow-down
              2
              ·
              1 year ago

              @magic_lobster_party I can’t believe someone wrote that. Incorrect answers do more harm than being useful. If the person asks and don’t know, how should he or she know it’s incorrect and look for a hint?

              • CloverSi@lemmy.comfysnug.space
                link
                fedilink
                arrow-up
                8
                ·
                1 year ago

                I don’t know about others’ experiences, but I’ve been completely stuck on problems I only figured out how to solve with chatGPT. It’s very forgiving when I don’t know the name of something I’m trying to do or don’t know how to phrase it well, so even if the actual answer is wrong it gives me somewhere to start and clues me in to the terminology to use.

              • seang96@spgrn.com
                link
                fedilink
                arrow-up
                8
                ·
                1 year ago

                In the context of coding it can be valuable. I produced two tables in a database and asked it to write a query and it did 90% of the job. It was using an incorrect column for a join. If you are doing it for coding you should notice very quickly what is wrong at least if you have experience.

              • magic_lobster_party@kbin.social
                link
                fedilink
                arrow-up
                4
                ·
                1 year ago

                Google the provided solution for additional sources. Often when I search for solutions to problems I don’t get the right answer directly. Often the provided solution may not even work for me.

                But I might find other clues of the problem which can aid me in further research. In the end I finally have all the clues I need to find the answer to my question.

                • preciouspupp@sopuli.xyz
                  link
                  fedilink
                  arrow-up
                  2
                  arrow-down
                  1
                  ·
                  1 year ago

                  How do you Google anything when all the results are AI generated crap for generating ad revenue?

                  • magic_lobster_party@kbin.social
                    link
                    fedilink
                    arrow-up
                    2
                    ·
                    1 year ago

                    Well then I guess I have to survive with ChatGPT if the internet is so riddled with search engine optimized garbage. We’re thankfully not there yet, at least not with computer tech questions.

              • psyspoop@kbin.social
                link
                fedilink
                arrow-up
                4
                ·
                1 year ago

                In my experience, with both coding and natural sciences, a slightly incorrect answer that you attempt to apply, realize is wrong in some way during initial testing/analysis, then you tweak until it’s correct, is very useful, especially compared to not receiving any answer or being ridiculed by internet randos.

              • cat@feddit.it
                link
                fedilink
                arrow-up
                2
                ·
                1 year ago

                Well if they refer to coding solution they’re right : sometimes non-working code can lead to a working solution. if you know what you’re doing ofc

                • FaceDeer@kbin.social
                  link
                  fedilink
                  arrow-up
                  2
                  ·
                  1 year ago

                  Even if you don’t know what you’re doing ChatGPT can still do well if you tell it what went wrong with the suggestion it gave you. It can debug its code or realize that it made wrong assumptions about what you were asking from further context.

          • QHC@kbin.social
            link
            fedilink
            arrow-up
            9
            ·
            1 year ago

            How is that practically different from a user perspective than answers on SO? Either way, I still have to try the suggested solutions to see if they work in my particular situation.

            • bionicjoey@lemmy.ca
              link
              fedilink
              arrow-up
              8
              arrow-down
              1
              ·
              1 year ago

              At least with those, you can be reasonably confident that a single person at some point believed in their answer as a coherent solution

                • bionicjoey@lemmy.ca
                  link
                  fedilink
                  arrow-up
                  3
                  arrow-down
                  1
                  ·
                  1 year ago

                  Better than knowing there’s some possibility that the answer was generated purely because the sequence of characters had the highest probability of convincing the reader that it seems correct based on the sequence of characters it was given as input (+/- a decent amount of RNG)

                  • FaceDeer@kbin.social
                    link
                    fedilink
                    arrow-up
                    3
                    ·
                    1 year ago

                    Still debatable, IMO. Human belief is stubborn and self-justifying whereas an RNG can be rerolled as many times as needed.

              • Greg Clarke@lemmy.ca
                link
                fedilink
                English
                arrow-up
                1
                ·
                edit-2
                1 year ago

                not for solving technical problems

                One example is writing complex regex. A simple well written prompt can get you 90% the way there. It’s a huge time saver.

                for generating prose

                It’s great a writing boilerplate code so I can spend more of my time architecturing solutions instead of typing.

          • focus@lemmy.film
            link
            fedilink
            arrow-up
            2
            ·
            1 year ago

            the good thing if it gives you the answer in a programming language is that its quite simple tontestvif the output is what you expect, also a lot of humans hive wrong answers…

        • displaced_city_mouse@midwest.social
          link
          fedilink
          English
          arrow-up
          9
          arrow-down
          1
          ·
          1 year ago

          There was a story once that said if you put an infinite number of monkeys in front of an infinite number of typewriters, they would eventually produce the works of William Shakespeare.

          So far, the Internet has not shown that to be true. Example: Twitter.

          Now we have an artificial monkey remixing all of that, at our request, and we’re trying to find something resembling Hamlet’s Soliloquy in what it tells us. What it gives you is meaningless unless you interpret it in a way that works for you – how do you know the answer is correct if you don’t test it? In other words, you have to ensure the answers it gives are what you are looking for.

          In that scenario, it’s just a big expensive rubber duck you are using to debug your work.

          • FaceDeer@kbin.social
            link
            fedilink
            arrow-up
            8
            ·
            edit-2
            1 year ago

            There’s a bunch of people telling you “ChatGPT helps me when I have coding problems.” And you’re responding “No it doesn’t.”

            Your analogy is eloquent and easy to grasp and also wrong.

            • displaced_city_mouse@midwest.social
              link
              fedilink
              English
              arrow-up
              1
              arrow-down
              1
              ·
              1 year ago

              Fair point, and thank you. Let me clarify a bit.

              It wasn’t my intention to say ChatGPT isn’t helpful. I’ve heard stories of people using it to great effect, but I’ve also heard stories of people who had it return the same non-solutions they had already found and dismissed. Just like any tool, actually…

              I was just pointing out that it is functionally similar to scanning SO, tech docs, Slashdot, Reddit, and other sources looking for an answer to our question. ChatGPT doesn’t have a magical source of knowledge that we collectively also do not have – it just has speed and a lot processing power. We all still have to verify the answers it gives, just like we would anything from SO.

              My last sentence was rushed, not 100% accurate, and shows some of my prejudices about ChatGPT. I think ChatGPT works best when it is treated like a rubber duck – give it your problem, ask it for input, but then use that as a prompt to spur your own learning and further discovery. Don’t use it to replace your own thinking and learning.

              • FaceDeer@kbin.social
                link
                fedilink
                arrow-up
                2
                ·
                1 year ago

                Even if ChatGPT is giving exactly the same quality of answer as you can get out of Stack Overflow, it gives it to you much more quickly and pieces together multiple answers into a script you can copy and work with immediately. And it’s polite when doing so, and will amend and refine its answers immediately for you if you engage it in some back-and-forth dialogue. That makes it better than Stack Overflow and not functionally similar.

                I’ve done plenty of rubber duck programming before, and it’s nothing like working with ChatGPT. The rubber duck never writes code for me. It never gives me new information that I didn’t already know. Even though sometimes the information ChatGPT gives me is wrong, that’s still far better than just mutely staring back at me like a rubber duck does. A rubber duck teaches me nothing.

                “Verifying” the answer given by ChatGPT can be as simple as just going ahead and running it. I can’t think of anything simpler than that, you’re going to have to run the code eventually anyway. Even if I was the world’s greatest expert on something, if I wrote some code to do a thing I would then run it to see if it worked rather than just pushing it to master and expecting everything to be fine.

                This doesn’t “replace your own thinking and learning” any more than copying and pasting a bit of code out of Stack Overflow does. Indeed, it’s much easier to learn from ChatGPT because you can ask it “what does that line with the angle brackets do?” or “Could you add some comments to the loop explaining all the steps” or whatever and it’ll immediately comply.

    • blueson@feddit.nu
      link
      fedilink
      arrow-up
      22
      arrow-down
      1
      ·
      1 year ago

      I honestly believe people are way overvaluing the responses ChatGPT gives.

      For a lot of boilerplating scenarios or trying to resolve some pretty standard stuff, it’s good.

      I had an issue a while back with QueryDSL running towards an MSSQL instance, which I tried resolving by asking ChatGPT some pretty straightforward questions regarding the tool. Without going too much into detail, I basically got stuck in a loop where ChatGPT kept suggesting solutions that were not viable at all in QueryDSL. I pointed it out, trying to point out why what it did was wrong and it tried correcting itself suggesting the same broken solutions.

      The AI is great until whatever it has been taught previously doesn’t cover your situation. My solution was a bit of digging in google away, which helpfully made me resolve the issue. But had I been stuck with only ChatGPT I’d still be going around in loops.

      • CloverSi@lemmy.comfysnug.space
        link
        fedilink
        arrow-up
        10
        ·
        1 year ago

        It really doesn’t work as a replacement for google/docs/forums. It’s another tool in your belt, though, once you get a good feel for its limitations and use cases; I think of it more like upgraded rubber duck debugging. Bad for getting specific information and fixes, but great for getting new perspectives and/or directions to research further.

        • blueson@feddit.nu
          link
          fedilink
          arrow-up
          1
          ·
          1 year ago

          I agree! It has been a great help in those cases.

          I just don’t believe that it can fullfill the actual need for sites like StackOverflow. It probably never will be able to either, unless we manage to make it learn new stuff without reliable sources like SO, while also allowing it to snap up these obscure answers to problems without burying it in tons of broken solutions.

      • Gamma@beehaw.org
        link
        fedilink
        arrow-up
        3
        ·
        1 year ago

        ChatGPT is great for simple questions that have been asked and answered a million times previously. I don’t see any downside to these types of questions not being posted to SO…

    • Elw@lemmy.sdf.org
      link
      fedilink
      arrow-up
      15
      ·
      1 year ago

      Exactly this. SO is now just a repository of answers that ChatGPT and it’s ilk can train against. A high percentage is questions that SO users need answers to are already asked and answered. New and novel problems arise so infrequently thanks to the way modern tech companies are structured that an AI that can read and train on the existing answers and update itself periodically is all most people need anymore… (I realize that was rambling, I hope it made sense)

      • focus@lemmy.film
        link
        fedilink
        arrow-up
        3
        ·
        1 year ago

        yes! this! is chatgpt intelligent: no! does it more often than not give good enough answers to daily but somewhat obscure ans specific programming questions: yes! is a person on SO intelligent: maybe. do they give good enough answers to daily but somewhat obscure ans specific programming questions: mostly

        Its not great for complex stuff, but for quick questions if you are stuck. the answers are given quicker, without snark and usually work