Elon Musk recently participated in an engaging Twitter Spaces discussion focused on the latest developments and open questions in artificial intelligence (AI). As one of the tech world’s most prominent thinkers on the transformative potential of AI, Musk provided unique perspectives.
The X space was hosted on Thursday, 21 December 2023, by Cathie Wood (prominent tech investor and founder of ARK Invest).
Musk has long been one of the leading voices shaping the public dialogue around artificial intelligence. From founding organizations like OpenAI and Neuralink to incorporating AI capabilities into Tesla vehicles, Musk understands the technology’s vast potential – as well as its risks.
Wood and Musk covered numerous topics fundamental to understanding where AI is headed next:
- The debate around open-source vs closed-source AI systems.
- Assessing if AI will improve truth and accuracy at scale.
- Key architectures like transformers and diffusion models.
- Comparing biological intelligence to rapidly advancing digital intelligence.
- Speculations on what achieving AGI (artificial general intelligence) would entail.
In this post, we’ll summarize Musk’s perspectives across all five areas and synthesize the key takeaways around the future of AI.
Open Source vs Closed Source: Enabling an AI “Truth Race”
The conversation began by revisiting Musk’s past involvement as co-founder of OpenAI and his original vision to create the nonprofit as an open-source counterbalance to closed systems like Google’s DeepMind.
Where does Musk stand on the open-source versus closed-source debate embroiling the AI community? His perspective remains nuanced, balancing both philosophical preferences and pragmatic assessments of progress.
Musk expressed frustration at how ironically, OpenAI has now pivoted from that open model to being “closed for maximum profit”. He reiterated his preference for open approaches that allow full transparency and auditability into AI systems.
Musk believes his company’s model ‘Grok’ aims precisely for this type of rigorous commitment to truth and neutrality. And the presence of such accurate, impartial AIs could indeed raise the bar for entire industries.
I generally have a bias in favor of open source…you can see that certainly with the X platform where we’ve open-sourced the algorithm. For Community Notes we open-sourced not just the algorithm but all the data as well so you can see exactly how a note was created.
However, Musk acknowledged the reality that closed-source AI retains a meaningful advantage currently in raw performance over open counterparts. He estimates closed source systems to be roughly 6 months ahead in capabilities simply due to greater access to private computing and data resources. In other words, while openness may provide more feedback loops for safety and ethical development, the pace of progress also matters deeply. And on that metric, closed models currently hold the lead.
That said,
Musk believes that as AI gets smarter and open-source models focus on being accurate and truthful, it could start a competition for truth, a “Truth-race”. This means that private AI providers might also have to focus on accuracy and truth, otherwise, people might stop trusting them.
I think AI truth-seeking arms race…there will be a competition for truth and people will tend towards the one they think is most accurate. So if there’s at least one AI aiming for maximum accuracy I think it pushes all AIs to aim for maximum accuracy.
Physical World Understanding Required for Truth-Seeking AI
while AI has improved a lot by learning from language data, Musk argues the importance of AI to understand the real world to be more accurate and make fewer mistakes. He gives examples of Tesla’s self-driving technology and their robot, Optimus, which need to truly understand how the messy and complex real world works.
Mastering just textual inputs alone likely won’t be enough:
“Tesla like really to to make full self-driving work you kind of need baby AGI in the car… because you need to understand reality and reality is messy and complicated.”
Tesla’s fleets of cars are gathering lots of images, sensor information, and driving situations. This could make Tesla’s AI data one of the most valuable. This real-world data could give Tesla an edge over other AI systems that don’t have access to such information. By combining this large amount of real-world data with advanced AI designs, we could make faster progress towards creating AI that can understand and learn anything that a human being can.
Transformers & Diffusion: Building Blocks for AGI
On the topic of specific machine learning architectures instrumental to future AI capabilities, Musk strongly advocates for the central importance of transformers. He states unambiguously that “pretty much everyone’s using Transformers”. While there are different versions of “Transformers” that use different settings, hyperparameter configurations and sparsity tradeoffs, transformer networks have clearly become a common base for most of the top-performing AI models in language, vision, robotics, and multimodal models.
In addition, Musk highlights diffusion models as another critical emerging technique. He hypothesizes that artificial general intelligence essentially involves finding the optimal combination of transformers and diffusion models. Active research is still ongoing into exactly how to composite those architectural blocks and weigh their relative contributions.
It really looks like AGI is some combination of Transformers and diffusion. .You really can’t do AI without transformers in a meaningful way.
As large language models continue rapidly scaling up parameter counts in pursuit of human-level intelligence, transformers and diffusion model advancements could provide a guiding software architecture blueprint.
Biological vs Digital Intelligence: The Crucial Ratios
Musk also discusses how he thinks about intelligence in a broader sense, not just specific AI methods. He often uses physics as a source of inspiration, breaking down problems to their most fundamental ratios. Through that lens, tracking the ratios between biological (natural) and digital (artificial) forms of intelligence acts as highly informative metrics.
Specifically, Musk calls out two key ratios:
Memory: Digital memory already exceeds biological memory in humans by over 100x by Musk’s estimates. The advent of smartphones alone outsourced much of our memorial function to silicon (computer chips) rather than our brains. we’ve started to rely more on electronic devices to remember things. The amount of storage space available on these devices keeps expanding exponentially.
Compute: While the raw computing capability contained within a fixed human brain is largely static, digital computing doubles approximately every year. We could soon be at over 1000x more computing power than our brains, making them the main source of intelligence.
These massive and ever-growing advantages accumulating on the digital side bolster Musk’s view that artificial intelligence (AI) will eventually surpass human intelligence. He acknowledges that humans currently have unique strengths in creativity and invention. However, he thinks AI will follow the same path as calculations and memory, which have moved from being mostly done by humans to being mostly done by computers. The speed and ability of digital intelligence to handle large amounts of information might make it hard for the slower, more complex process of biological evolution to keep up in the long run.
Achieving AGI: Mastering Money & Comedy
As humans spend more time interacting with artificial agents like Siri, Alexa, and generative AI, expectations around conversational ability and general reasoning continue to grow. While we’ve made a lot of progress in making AI good at specific tasks, it’s still a big challenge to create an AI that can understand and learn anything that a human being can.
Musk though does call out two emerging benchmarks he’s tracking as key indicators of progress towards AGI:
- The Money Test: Also referred to as the “Han Solo test”, whether an AI system can successfully complete practical real-world tasks like filling out expense reports, paying bills and fees, filing taxes etc. Right now, humans don’t have many examples of machines that can handle these kinds of responsibilities.
- The Comedy Test: Demonstrating a mastery of humor, wit, sarcasm and other forms of implicit language remains difficult for machines. If an AI can reach human levels of comedic competency and even tell original jokes, it suggests a deeper understanding of culture and communication.
In Musk’s estimation, developing AI capable of exhibiting both meaningful financial responsibility and creative comedic expression would constitute major milestones towards artificial general intelligence.
Conclusion: The Coming AI Truth Renaissance
Elon Musk’s lasting influence on the future of AI cannot be understated given his dual roles as both a builder and thought leader. Even as CEO of both SpaceX and Tesla, he finds time to engage in substantive discussions on how artificial intelligence might shape our future world.
And despite his reservations around the risks of uncontrolled advanced AI, Musk remains an optimist about the net progress machine learning can catalyze – a true “truth renaissance.” Between open frameworks prioritizing accuracy and safety, powerful architectural advances like transformers and diffusion, and the scalability of digital intelligence overall, Musk sees the potential for AI to elevate fact over fiction.
By responsibly incorporating artificial intelligence into products and research initiatives, Musk aims to push all of Silicon Valley to maximize transparency, auditability, and truth-seeking. The knowledge accrued in humans and machines collectively could propel civilization to incredible new heights.
Paraphrasing Musk’s words:
There will be a competition for truth, and people will tend towards the one that they think is most accurate… as soon as you have at least one AI that is aiming for maximum accuracy I think it pushes all others to aim for maximum accuracy.
The path ahead remains unclear, but if leaders across technology echo Musk’s commitment to openly collaborate around building beneficial AI, the future looks bright. Our truth renaissance may have already begun.
So what do you think – will AI lead us into a new age of transparency and understanding? Or will it absorb all our knowledge into mysterious, incomprehensible systems? Share your thoughts in the comments below!
Still have some lingering questions?
Is Elon Musk concerned about AI becoming too powerful?
Yes, Musk has consistently voiced concerns about the risks of uncontrolled advanced AI potentially causing harm. But he remains optimistic that embracing openness and transparency will allow AI progress to be directed towards benefiting humanity with technologies like self-driving vehicles and intelligent assistants.
What is the current state of self-driving car technology?
While fully autonomous vehicles still face challenges, Musk notes that Tesla AI already has extremely high reliability in optimal road conditions like highways. They are steadily improving performance in more complex city navigation to reduce interventions to extremely infrequent levels.
How soon could AI reach human-level intelligence?
Musk declined to give an exact timeline but pointed to his previous forecasts expecting AGI within the next decade as key innovations in transformers, generative diffusion models, and real-world data from robots accumulate. The exponential pace of progress makes precise predictions difficult though.
What breakthroughs need to happen for AI to keep advancing?
Musk believes combining gains in model architectures like transformers and diffusion with mass amounts of real-world physical data will lead to AI systems better grounding their learning in the messy complexity of reality. Relying solely on language data may hit limits.
Does Musk think AI will become conscious?
This remains an open philosophical debate amongst researchers. Some believe machines could one day attain strong notions of self, introspection, and subjective experience akin to humans. Musk did not explicitly speculate either way during his discussion with Cathie Wood.
