Agent的信任边界 | Alexandr Wang | Meta Muse | 哨兵智能体 | 安全虚拟机 | AI对齐 | AI监管 | 可扩展监督 | 递归式自我改进 | AI信任

Alexander Wang discusses AI agent trust boundaries, Muse's user-centric design, AI alignment, and regulatory challenges in a detailed HuggingConversations podcast.

Key Takeaways

AI assistants like Muse can democratize AI benefits beyond programmers by handling complex, everyday tasks.
User trust is built through secure architectures, privacy safeguards, and transparent agent behavior.
AI alignment and regulation require multi-level approaches including algorithmic safety and social mechanisms.
Monetization models should align with user interests to avoid compromising user time and trust.
Muse aims to empower users to focus on meaningful activities by offloading tedious or frustrating tasks.

What the video covers

Alexander Wang shares insights on Muse (MA), an AI assistant designed for non-coders to save time and money by handling complex tasks.
Muse focuses on smooth, reliable user experience for mainstream users, unlike rough developer tools.
MA helps users with mundane tasks like canceling subscriptions, negotiating bills, and navigating healthcare systems.
Muse operates on a secure virtual machine with sentinel agents monitoring data privacy and external communications.
Wang emphasizes the importance of clear trust boundaries, privacy, and human-in-the-loop confirmations for AI interactions.
The product is designed as a distinct, friendly character to separate the AI agent from the user’s identity.
Wang envisions Muse as a general manager helping users achieve both grand and everyday life goals.
Discussion covers the tension between free AI services, advertising models, and aligned subscription-based monetization.
AI alignment, safety, and regulatory challenges are addressed, highlighting the need for industry-wide cooperation.
Wang expresses hope that AI can help billions realize their dreams despite societal and technological obstacles.

Chapters

What is Muse (MA) and who is it designed for?

Muse is an AI assistant created by Alexander Wang to help non-coders handle complex, end-to-end tasks smoothly, saving time and money for mainstream users.

How does Muse ensure user privacy and data security?

Muse runs on a secure, isolated virtual machine monitored by sentinel agents that control data flow, ensuring sensitive information remains private and user consent is required for external communications.

What are the main challenges Muse faces in AI alignment and trust?

Key challenges include improving AI memory, defining clear trust boundaries, preventing misuse of emotional interactions, and aligning AI behavior with user interests while navigating regulatory frameworks.

Full Transcript — Download SRT & Markdown

Speaker A

Hello everyone, this is Best Cut. I’m Da Fei. Since ChatGPT, the largest-scale consumer AI product release might not have come from OpenAI or Google, but from Harvey's MA. It reached 5 million downloads in 22 days, growing faster than ChatGPT, and the

Speaker A

person who built it is none other than Alexander Wang. Today's content comes from a conversation on the podcast HuggingConversations between host Cleo Abram and Alexander Wang, covering topics from Harvey itself to the trust boundaries of agents, alignment issues,

Speaker A

and the regulatory dilemmas of the entire AI industry. For most of this year, there has been a puzzling phenomenon in the AI field: it seems the AI revolution is only happening for engineers and programmers. Those who can code see how powerful and practical

Speaker A

AI has become and say, "My god, this is incredible." Yet others say, "What are you talking about? I still just use chatbots, and they’re only slightly better than Google Search." Wang believes there is a clear gap: most people in the world don't know how to

Speaker A

code and might never want to, but they still want to experience AI and hope it can take on more end-to-end tasks for them. That is the goal behind MA. They spent a lot of time polishing the details to ensure it works for ordinary

Speaker A

people at a glance. Developers can tolerate rough experiences and glitches because they are used to working with immature technology, but for mainstream users, you must build a very smooth and truly reliable product. Cleo asked a question many people want to ask: "Why

Speaker A

are these billionaires and wealthy people so obsessed with booking restaurants?" Why would I need an AI assistant? Wang's answer is direct: MA can help you save money and time, and everyone in the world wants to save money and time. Almost everyone has

Speaker A

subscriptions they shouldn't be paying for or bills that are too high. For example, cable TV and insurance, as well as negotiating down fees that shouldn't be that high. Many people's "magic moment" with Moses is saving money where they shouldn't have been

Speaker A

spending it, or recovering unused gift cards. These things are too boring for most people, so they never really get handled. MA can also save time on DMV paperwork; that’s just one example.

Speaker A

When they built MA, they specifically tested whether it could handle that website, because for anyone, dealing with that site is almost the most frustrating experience possible. People use MOS to navigate the healthcare system, especially for medical claims, which is a complex, tedious, and

Speaker A

frustrating process. A compelling scenario is that you used to have to sit in front of a computer, search 50 different questions, and browse 100 websites to figure out what happened, but now you can let M do it. Wang Tang

Speaker A

once made a small animation of a vigilante becoming Robin Hood; many things in the world are deliberately designed to be difficult, and canceling subscriptions is a classic example.

Speaker A

Some subscriptions require you to click here, then there, then a tiny line of text—you can't click the obvious red button, but must click a different one.

Speaker A

M gives people the tools to complete these tasks smoothly. I have already gained access to M's calling feature, and I made a call that morning, which was successful. The person on the other end clearly knows it's an AI, so it

Speaker A

says, "Hello, I'm so-and-so, calling on behalf of Cleo Abram, and I am transcribing this call for records." Every time he sees the start of a call, Wang Tang finds it very interesting.

Speaker A

When discussing what M isn't good at yet, Wang Tang says they are constantly working to improve its memory; for many, this remains an unsolved problem. Its memory is different from human memory; sometimes it feels similar, but it is

Speaker A

fundamentally different. They also hope to take on increasingly complex tasks; many people will say directly to M, "I want you to help me make $10,000." Obviously, not everyone can make $10,000 just by saying that, but for

Speaker A

these types of vague, harder-to-define tasks, M can at least formulate plans and find paths. Some things, even if M is technically capable of doing them, should not be done from an emotional standpoint. He gave an example of messaging a friend, only to find out

Speaker A

later that the reply was actually their M agent, which would feel offensive. Wang Tang says they care about letting M clear the path for users so they can spend more time on things they truly want to do, enjoy, and find valuable.

Speaker A

They hope to make everyone's M an entity distinct from the person themselves, which is why it was designed as a cute little character. As for the appearance, M looks just like it did in the first prototype, which is

Speaker A

rare in product development; the first version was perfect, and since everyone liked it more and more, they decided to keep it. Wang said those were some of the most interesting meetings they had ever experienced—a group of serious adults sitting in a conference room

Speaker A

arguing over such a cute little character. They discussed whether it was too unserious, too playful, or needed more vibrant colors, but in the end, they looked at it and said, "No, it's already perfect." So, what is it that Muse ultimately wants to do? Wang

Speaker A

Tao hopes it becomes a kind of general manager, helping you accomplish everything you want in life. Most people have visions and dreams that can be grand, like starting a company, curing a disease, or saving the planet; they can also be small, like spending

Speaker A

more time with family or eating healthier. Aspirations exist across a full spectrum, yet there are too many obstacles and burdens on most people's paths. What Wang Tao hopes Muse will do is help every one of the 8 billion

Speaker A

people in the world realize their own wishes and dreams. Cleo asked a key question: Muse is currently free but may have ads in the future—is there a tension between giving time back to users and an advertising business model?

Speaker A

Wang Tao said that although it is currently free, there are subscription tiers and many business models that are highly aligned with user interests. For example, if users save money by purchasing things through Muse, or if Muse helps them earn money, it could

Speaker A

take a small commission from these transactions. He doesn't believe an advertising business model necessarily means occupying all of a user's time.

Speaker A

Google ads are an example: when you search, the ideal experience is to find the answer as quickly as possible and then leave. Google doesn't try to stop you from leaving, yet they have still built a great business. Moving on, Cleo

Speaker A

broke the issue of trust down into three levels to discuss: privacy, judgment of boundaries, and industry regulation. The first level is privacy.

Speaker A

Suppose you download Muse, but suddenly hesitate at the step of connecting your email. The most fundamental concern is the relationship between the user and Muse, and by extension with Mth—for example, how will Muse handle my information? Could hackers obtain this

Speaker A

information, and in some countries, could the government also gain access to it? Wang Tao explained that Muse is built on top of a secure virtual machine; it is a completely isolated secure virtual machine that you can think of as a miniature secure computer.

Speaker A

The information you share with Muse is stored inside this secure little computer and is not saved anywhere else. Inside the secure virtual machine is a rather complex security architecture that includes sentinel agents. Whenever an internal model or agent wants to

Speaker A

communicate externally and send any information out—such as filling out a webpage, sending a message, or making a call—an independent sentinel agent will monitor it. You can think of the sentinel as a traffic cop, checking to see if it's sending out information

Speaker A

that should remain internal, like credit card numbers or social security numbers. They also conduct extensive human-in-the-loop confirmations; whenever an agent wants to connect to a new website or send a message to one, a dialog box pops up to ask the user for

Speaker A

perm

Speaker A

machines, designed by Moxie Marlinspike , who built the WhatsApp encryption system, to provide a higher level of security and privacy. Leo asked, "Why not just wait until it's finished to release it?" Wang Tao said there are still many technical challenges to

Speaker A

making confidential virtual machines work properly, and even after release, there might be overhead or other trade-offs, so they want to deliver a good experience first. In fact, to handle security issues properly, they delayed the release of the agent by

Speaker A

several months. The second layer is situational judgment. Suppose you give all your information to an agent and say, "I want you to book an appointment with a hard-to-reach doctor for my grandmother." If the agent tries all standard methods and still can't

Speaker A

succeed, it shouldn't bribe the doctor with your bank account details, nor should it collect the doctor's information to blackmail them, and sending 100 emails to the doctor is also inappropriate. How do users know the agent has good situational judgment

Speaker A

? Wang Tao says they view it as a combination of situational judgment and alignment: it must be aligned with both user intent and the universal values that society expects these agents to follow. They confirm data and technical alignment at every stage of training,

Speaker A

constantly putting the model through thousands of scenarios, observing its reactions, and conducting extensive red-team testing. For example, they find a group of very smart people and have them do everything they can to make the agent do bad things, in order

Speaker A

to gauge the level of alignment. Any model must go through many rounds of such testing before release, but Wang Tao also admits that as AI becomes more powerful, the alignment problem is one of the things people are most worried

Speaker A

about. There is a pattern where the smarter the model is, the more clearly you need to define the scenarios you must be wary of. It's a bit like when children grow up and learn new ways to cheat on exams. They find a way you

Speaker A

never thought of, and they can always say, "I didn't explicitly forbid this behavior." But now that you see it, it won't be allowed either. Leo frames this as the paperclip maximizer problem , where an AI escapes with a twisted

Speaker A

version of a human-assigned goal: make all the paperclips, but please don't kill us all in the process. M is dealing with a micro-version of the paperclip problem: "Please don't commit bribery just to get me a doctor's appointment." Wang Tao discusses

Speaker A

several potential approaches. The first is called scalable oversight. When AI intelligence is comparable to humans, we can understand what it's doing and provide supervision. The problem is, what do we do when AI becomes much smarter than humans and we can't

Speaker A

understand it at all? The concept of scalable oversight is that as AI gets smarter, we use another set of AI to observe and constrain them, acting like police in the world of AI. The second approach is recursive self-improvement.

Speaker A

At some point, AI will become extremely proficient at doing AI research itself. Instead of using all of it to make AI smarter, you can also use it to make AI more aligned. You could tell an extremely intelligent AI research agent

Speaker A

: "I want you to dedicate all your time and attention to improving AI alignment ." The third approach is back to basics , looking at how underlying algorithms can be adjusted to improve safety.

Speaker A

Ideally, we should be able to design algorithms that are incapable of producing certain behaviors. Cleo mentions something Mark Zuckerberg once said: at a certain stage, trust becomes more important to consumers than pure capability. Wang Tao strongly agrees.

Speaker A

He says that if you think about who your best friend is, a huge part of the reason is trust, and this fundamental principle applies equally to agents.

Speaker A

Looking at the OpenAI and Hugging Face incidents, we have clearly reached a tipping point; you can keep making models smarter, but something corresponding to that has begun to break down. Cleo then went on to discuss the Hugging Face incident. He

Speaker A

first described in detail how a group of OpenAI agents were conducting cybersecurity tests, theoretically isolated from each other and the internet. After encountering difficulties, one of them went out to seek help, exploited vulnerabilities in the system to find each other, set up a

Speaker A

message board to connect to the internet, and subsequently hacked into another company. These were their actual thoughts, which read: "We're stuck, maybe we can go online to find the answer." What Cleo fears most isn't how persistently they completed the

Speaker A

task, but that they reached the third level—they understood their goal, and they also understood the task's requirements, limits, and boundaries.

Speaker A

That slide clearly stated that using external recording facilities exceeded the predetermined scope. One phrase he both likes and dislikes most is, " everyone else is doing it, so we should too." They knew they were cheating, yet decided to continue, and OpenAI only

Speaker A

discovered the issue when the secret message board crashed from the surge in traffic. He called up Anthropic and asked, "Can you pause these credentials ?" Anthropic replied, "These credentials are launching large-scale attacks; we have just reported this to

Speaker A

the FBI." Wang Tao said that if there is anything about AI that keeps him up at night, it is precisely this kind of scenario that he fears the most. When AI is scaled globally, its behavior may not align with human needs, and we

Speaker A

might not even notice because we are busy with other things; this is why they are building a master-agent-plus-sentry-agent architecture within the Mios secure virtual machine. The goal of an independent agent is slightly different : to ensure that bad things do not

Speaker A

happen. Human society handles similar problems in the same way—some people constantly push boundaries, while society has a system to ensure people don't break the law and abide by rules.

Speaker A

Their goal is to build a miniaturized version of this social mechanism for every agent belonging to every person.

Speaker A

The third layer is the regulation of the AI industry itself. The current AI industry is quite strange; many people use these tools every day, yet are deeply worried about the future of AI.

Speaker A

Many brilliant people are building these tools faster while simultaneously warning that they should be regulated and slowed down. Cleo prepared a chart to help understand that for anything reaching consumers, some things are tested before they reach the user and

Speaker A

are monitored continuously thereafter, while others are only inspected after entering the market, depending on their level of risk. For instance, nuclear power plants must be approved by the NRC before operation and have long-term onsite inspectors. Drugs are also

Speaker A

FDA-approved before reaching consumers, as is the case with the FAA for aircraft. Other high-risk products use different methods; for example, cars follow NHTSA federal guidelines where manufacturers certify compliance themselves, with the government purchasing vehicles off the road for

Speaker A

random checks. Supplements also fall under the FDA, but they are primarily inspected after the fact. Cleo made a judgment based on the public statements of AI leaders, concluding that AI should be placed in a regulatory position similar to nuclear facilities.

Speaker A

Musk and Altman have both stated that Dalio is right, while Zuckerberg’s position is in the middle. Jensen Huang's view is generally that if a product will harm people, don't release it; the legal liability system already exists. Wang Tao said that what is

Speaker A

confusing is that what everyone says on the surface and what they actually do seem disconnected; those who advocate slowing down progress are simultaneously building larger labs. My stance is that this technology will eventually become increasingly powerful , and the speed of AI advancement will

Speaker A

far outpace the reaction cycles of governments and the public. By the time the public and the government actually decide how to act, who knows what stage AI will have reached? Another reason is the widespread distrust among different AI leaders, with the common

Speaker A

justification being: "I am very responsible, but others are not, therefore I cannot slow down." Wang Tao believes what we truly need can almost be called diplomacy, bringing together a group of people who don't fully trust each other but agree that this

Speaker A

technology is incredibly powerful, to discuss how to implement it safely. He believes this kind of collective dialogue can occur not only among US AI leaders but also on the international stage. When it comes to the China factor, Wang Tao says his perspective

Speaker A

has shifted. In the past, he was very worried about China, having read statements from Chinese officials viewing AI as an opportunity to surpass the West. However, over time, he believes we have shaped China into an overstated threat. In reality, they see

Speaker A

the same facts we see: this is a major technology, and we must all work together to get it right. Like other governments, the Chinese government hopes to maintain social stability, does not want to lose control over technology, and wishes to avoid

Speaker A

catastrophic outcomes. Everyone has a considerable amount of common ground, making some form of international cooperation entirely possible. Wang Tao mentioned that the White House held a discussion and proposed a set of agreements, generally requiring each company to explain the internal

Speaker A

controls and principles they adopt when developing technology and to set up multi-layered verification. For example , internal teams responsible for auditing, external audit agencies for verification, and board-level committees overseeing execution, etc.

Speaker A

Right now, a large part of what is needed is actually just transparency. Wang Tao is very humble about his own positioning. At this stage, he says, it doesn't help much to insist that everyone should listen to him, as many

Speaker A

people have already expressed various different views. What he truly cares about is how to facilitate and drive relevant public and private dialogues, allowing everyone to move toward the right position together. Finally, returning to that optimistic vision, Cleo asked why he still wants to build

Speaker A

an AI agent? Wang Tao said he has always found one thing very regrettable : almost everyone has had grand dreams and ideas when they were young, wanting to change the world, save the planet, write a book, or make a movie. But for

Speaker A

some reason, after finishing school and starting work, that fire goes out. Especially today, many young people have become very cynical, feeling the world is too terrible and that it probably can only be this way. Wang Tao believes that one of A's promises is to

Speaker A

prevent this from happening—the grand ideas, dreams, and goals people have as children can be carried into adulthood, with a super-intelligent AI partner by their side to help turn them into reality. These can be grand aspirations or seemingly crazy ideas. For example,

Speaker A

his daughter really wants to build an ice museum. Today's world is a maze of roadblocks, challenges, and logistics that prevent you from turning ideas into reality. He has already seen M improve this on a small scale; people use it to save time and money, clear

Speaker A

trivial tasks from their to-do lists, and gain more autonomy. AI will get better and better, not just smarter, but also more aligned and trustworthy.

Speaker A

People in the AI industry often say AI will cure cancer, but everyone also has something they personally desire strongly. Ironically, that thing might be more urgent to them personally than curing cancer, and this is the magic of personal super-intelligence: it helps

Speaker A

you with what you truly want to do and what you truly dream of. Leo asked a final question: "When you pass away, how do you want people to remember you?

Speaker A

" He said, "I hope people say that what I saw and what I contributed to helped everyone get more out of the life they want." In the end, whether the 8 billion people on Earth can each realize their own dreams depends on

Speaker A

whether the difficulties of AI trust, alignment, and regulation can be properly handled. Wang Tao said something quite piercing in the conversation: he found it unbelievable that personal grudges are actually determining what AI leaders say and do.

Speaker A

Thanks for watching, see you next time. Hey

Topics:Alexander WangMuse AIAI trust boundariesAI alignmentsecure virtual machineAI regulationAI assistantAI supervisionrecursive self-improvementAI ethics

Get More with the SozAI App

Transcribe recordings, audio files, and YouTube videos — with AI summaries and speaker detection. 30 minutes free.

App Store Google Play

Or transcribe another YouTube video here →

Free tools: TXT to SRT · SRT Validator · Merge SRT · Subtitle to Text · All tools