Speaker A
And that's it. These skills are now installed and available to this researcher agent. So, this did a better job than just asking it to look for poison because I asked it to search for memory and then limit the results to
Wanderloots shows how to upgrade a Hermes researcher agent with custom skills, Zotero library access, and Consensus academic paper search, then tests it on agentic memory poisoning research.
Key Takeaways
- Equipping a specialist agent with custom skills and external tools (Zotero, Consensus) significantly improves research output quality and grounding.
- Multi-agent setups benefit from a separation of duties: researcher produces, orchestrator checks quality, librarian ingests, and the human gives final approval.
- Asking the agent how to use a skill is an effective way to learn its workflow.
- Targeted, specific queries with constraints yield better academic results than broad searches.
What the video covers
- The video upgrades a specialist 'researcher' agent inside Hermes bot mode, using the same researcher created in the previous video.
- Three skills are installed: an LLM wiki research skill (with research lenses like academic, current reality, and combined), a Zotero connection, and a Consensus academic paper discovery skill.
- The researcher pulls the creator's existing sources from Zotero, current practice sources, and new academic literature from Consensus to compare evidence.
- The workflow hands findings to an orchestrator agent for quality assurance, which can send work back to the researcher if it is not good enough.
- Approved findings are passed to a librarian agent that ingests them into an LLM wiki knowledge management system, with the researcher itself barred from writing to the wiki.
- The creator asks the researcher to explain its research lenses and uses its own advice to shape the query, favoring original sources and looking for counter-evidence.
- A test on agentic memory poisoning demonstrates that targeted queries (limiting results to papers on memory harms) produce better results than generic searches.
- The researcher accesses the Zotero library running on the desktop even though the agent runs inside Docker.
- Highlights from Zotero can be synced to Obsidian, linking quotes back to sources for reading and tracking.
- The final report enters a 'needs review' status, awaiting the creator's approval before a new wiki concept on agentic memory poisoning is added.
Chapters
- 00:00Introduction and installing the researcher's skills
- 01:36Planning the upgrade: sources, orchestrator, and librarian workflow
- 02:34Recap of the basic team and the three new skills
- 05:01Enabling the LLM wiki research skill and exploring research lenses
- 09:01Connecting Zotero and syncing highlights to Obsidian
- 11:22Sponsor segment and setting up the Consensus MCP
- 12:46Running a combined research task through the orchestrator
- 17:11Reviewing the report and approving it for the LLM wiki
Full Transcript — Download SRT & Markdown
Speaker A
paper that harm memory. Hermes bot mode lets you create specialized agents that improve your tasks both solo and as part of a team of agents. But more bots doesn't necessarily mean better. Today, I want to level up one specialist agent
Speaker A
to see what it can do and how it can boost the rest of the team both now and in the future. Hi, my name is Callum, also known as Waterloots, and welcome to today's video on upgrading a specialist
Speaker A
agent inside of Hermes. I'm focusing on upgrading the researcher first because it's a foundational teammate that can improve the quality of the rest of the team, but this process applies for any agent in any specialization. It also works with agents outside of Hermes as
Speaker A
well. To upgrade the researcher, we're going to give it a reusable research method, access my existing research in Zotterero, and connect to Consensus as an external tool for specialized academic paper discovery. Then, we'll work in a group chat to tie it all
Speaker A
together. Now, let's upgrade our researcher. In the last video, I gave an introduction to Hermes bots. So, if you're interested in learning more about what those are and how they work, I recommend checking that out. Today we're going to work with the same specialist,
Speaker A
the researcher that I created last time and actually upgrade its skills and capabilities. So Tarot brings in the work that I've already collected.
Speaker A
Current sources help it understands what tools and organizations are doing now. And new academic literature through consensus gives it research finding to compare with that current practice. The new research skill is what ties all of this together. It helps the researcher
Speaker A
inspect the sources, compare the evidence, and then explain the disagreements or gaps. All of those findings that were produced by the researcher and corroborated between the different sources are then handed off to the orchestrator. And this is intentional. The orchestrator can do
Speaker A
that quality assurance check before passing it off to me for final review. Then we can take that and hand it over to the librarian to ingest it into our LLM wiki or the knowledge management system that I give to my agents. I
Speaker A
thought that a good test would involve agentic memory poisoning as this is a topic that's becoming more and more important. So, we're going to take a look at where the evidence agrees, disagrees, and how we can fill the gaps
Speaker A
between them using this researcher agent. If you find this video helpful, please like, hype, and subscribe as I really appreciate your support a lot.
Speaker A
I've also put together a free Patreon article linked below that'll include the skills that I've created today that you can access for free. Hopefully that helps you speed up your own setup and start to see how these pieces can come
Speaker A
together to improve your own workflow using Hermes bots. This is where we left off with the last video. I created a librarian, a researcher, and an orchestrator. And then I put all three of them inside of this group chat here. I was able to
Speaker A
message the orchestrator who's able to delegate to the researcher to produce a task. The orchestrator checks the quality of it and if it's not good enough, passes it back to the researcher to go again and if it is good enough, it
Speaker A
passes it to me for approval to enter into the library with the LLM wiki. But this was just a very basic researcher. I didn't customize any particular skills with this. I just customized the soul or the personality and made sure it had the
Speaker A
built-in skills it needed to conduct research and ground that research. What we're going to do today is we're actually going to equip the researcher with some custom skills and access to external tools that will significantly enhance the quality of the reports that
Speaker A
it's able to produce so that when it's passing it back to the orchestrator, there's more context, there's more grounded citations, there's better sources so that when we eventually ingest it into the library, into the LLM wiki, into our trusted knowledge
Speaker A
management system, we know that the quality is significant. So, there's three skills we're going to introduce today. We're going to level up the ability to produce research in and of itself. So, this is the first skill, the LLM wiki research. And this one
Speaker A
basically just introduces different lanes or lenses that we'll be able to do research through, whether it's academic, current reality, or some combination of them. But I'll go through this more once I've installed the skill. Then we also have a Zotterero connection, which is
Speaker A
going to give my research specialist the ability to access my Zotterero library. So Tarot is a reference manager system where you can store PDFs and other sources and then it helps you build out a bibliography. So I'm going to connect
Speaker A
this to Hermes with the researcher. So it's able to pull from my own existing academic research and other sources. For example, like this one on memory poisoning that we'll get into in a little bit. And then the third skill I
Speaker A
want to bring in is the consensus academic path and how this also connects directly to Zotterero. So we can go back and forth between our sources that we're storing in Zotterero, the research agent that we can leverage in consensus and
Speaker A
our ability to connect all of this in Hermes to a single researcher agent. So let's get that set up.
Speaker A
The first step is to install the skills. So I'm going to copy these three skills.
Speaker A
I'll link the ability to download those in the top comment in a free Patreon article. So then we can go over to our Hermes profile, go to specifically the researcher profile. Then we can go down to skills over to research and then
Speaker A
paste these three skills in there. And that's it. These skills are now installed and available to this researcher agent.
Speaker A
I'm going to say, can you please enable the LLM wiki research skill? That will be the first one that we will enable. So basically what we just did is we created this specialist LLM wiki research skill and by adding it to the skills folder
Speaker A
inside of the researcher profile I was able to direct the researcher bot or profile the Hermes agent to load that specific skill and it will use that workflow for relevant research. So it will choose the appropriate research lens load the grounded citations for
Speaker A
retrieval and source handling and provide a readable report and source handoff. It will not directly admit to the LLM wiki. So actually in the previous video I disabled the LLM wiki skill for the researcher specifically so that it doesn't accidentally try to
Speaker A
write to the LLM wiki since that is the responsibility of the librarian only. Rather than me walking through the skill for you directly I thought why don't we just ask the researcher what are the different ways we can research. We have
Speaker A
a balanced research which mixes primary sources with secondary context. This is practical. We have academic which will prioritize papers, reviews, standards and official technical material. This is where we'll get into consensus. We have current reality which is official
Speaker A
records documentation. This is more trying to get into state-of-the-art. And then we can get combined which is where we can run two different passes. We can have the researcher run one pass where it does academic and another current
Speaker A
reality and then maps the academic or peer-reviewed sources to the current reality. So one of the best ways that you can use these agents more effectively is to ask it how to use it.
Speaker A
So I said is there anything else I should know about this skill that will help me understand how to use it? So it says give a specific question. Favor original sources. look for counter evidence and in the team setup it's
Speaker A
going to return it to the orchestrator which is the manager agent that we've built. It also depends on the grounded citation skill for citation handling which if we go back to capabilities for a moment we can see we have grounded
Speaker A
citations here. So basically this just brings in the ability to add footnotes and track citations as it's doing research. And that's a built-in skill that we are then enabling as part of this new skill. So we can just leverage
Speaker A
what already exists inside of the citation skill. Okay. And now let's enable our second skill.
Speaker A
So Zotterero is kind of like a personal research assistant. It's a reference manager. It's a way to collect, organize, annotate, site, and share research. You can see I have this little connector up in the top here. I'm able
Speaker A
to directly save sources to my Zotterero library, which looks like this. You can see here, I just clicked on that, and it added it to this source right here, and there's a copy of the web. But this also
Speaker A
works for papers as well, which is one of the reasons why I'm talking about it today. It's super easy to collect things. You can organize it however you want. It brings in all citations, creates bibliographies, and you can sync
Speaker A
across devices, too. And it's also open source, which is great. So, we can see here that we have now enabled the waterlero readonly skill. It can search and inspect existing local library with readon, never modifying.
Speaker A
So, I just put this in for my own personal testing system. In the future, I will change this to be write as well.
Speaker A
This is just a safety mechanism that I don't want it to go through and start overwriting my reference manager that I've been using for years now. So again, let's ask Hermes to explain how this works. So this skill comes with some
Speaker A
scripts involved. It's called deterministic tooling. And basically what that means is it will validate the information that's coming from Zotterero to make sure everything aligns correctly. So we can give the researcher a bounded search task. It will run the
Speaker A
script to connect to Zotterero. It finds the items. It saves a copy in a research workspace under RAW, which is connected to my LLM wiki. And I talk about that a lot more in the trusted LLM wiki videos.
Speaker A
If throughout this video you're confused at all about what an agentic librarian is, what an LLM wiki is, and how we can bring structured knowledge and trusted knowledge into our system, I recommend checking out this video here. But once
Speaker A
it saves it, it then validates it and then presents it as usable evidence. What we need to do now is we need to go to Zotterero, which you can download for free. And under settings, we need to make sure we have allow other
Speaker A
applications on this computer to communicate with Zotterero. And then it gives the specific endpoint that will enable Hermes to connect to it. And I want to note as well, if you use Obsidian, you are able to connect Zotterero to Obsidian. You're able to
Speaker A
sync highlights that you make here. For example, something like this would sync this particular quote into Obsidian. So, it's also a great way for you to read academic papers, read other sources, and track your quotes and link them back to
Speaker A
sources. And that's one of the main reasons why I wanted to bring this into my researcher because I want the researcher to be able to check those sources and then use it as part of its research evidence before passing it back
Speaker A
to the orchestrator. But now that we have enabled the local API, why don't we test it?
Speaker A
Can you please check to see what sources we have on agentic memory poisoning? So I'm going to approve the ability to run this particular skill, the one that's using the Zotterero, and I'm going to allow this session. Okay, there we go.
Speaker A
So the test has been completed. We can see that it found one direct relevant source from the broad query of memory poisoning poison once exploit forever.
Speaker A
That's the exact one we have open right here. So it did miss one called hijacking agent memory. So I can tell the agent to broaden its search to go beyond just the titles. It just went through and did a title search for our
Speaker A
quick test. And we go over to my folder. We can see it created a little report with its findings. So that's great.
Speaker A
Basically what we've just shown is the researcher has the ability to access Zotterero which is running on my desktop. even though the agent itself is running inside of Docker. And what we can see too is it also sent the result
Speaker A
to Orchestrator for review to make sure that it met the quality standard. So that's what we set up in the last video.
Speaker A
Orchestrator replied back and said, "Yes, we agree. This seems legit." So we now have the ability for our researcher agent to query our existing Zotterero library. The next step is to expand what goes into that Zotterero library in the
Speaker A
first place. And that brings me to the third skill that we've installed, the consensus academic discovery.
Speaker A
So, I've just told the researcher to please enable the third skill, the consensus skill. And we can see that it says it supports bounded academic paper discovery through consensus while treating the summaries as discovery only. So, the skill creates a search
Speaker A
with clear inclusion, preserves useful paper metadata, records limits, requires following the actual link to inspect the original paper before making any claims, and it doesn't modify Zotterero or the wiki. So, in the future, I might change my librarian skill to be able to go pull
Speaker A
that original paper and actually file it in Zotterero, but that's for another day. Let me know if that's something you might be interested in. And I'm happy to make a video that explains more of that process.
Speaker A
A quick thank you to Consensus for sponsoring today's video. Consensus helps me discover papers on the topics that I'm most interested in and actually gauges the quality. You can use Consensus for free in the web app or through MCP, which I'll show you in a
Speaker A
moment. The MCP connection lets you search directly with consensus and you can get access to papers even without a paid account. I've been using it to explore topics like memory poisoning like the one that we're going to continue exploring today through Hermes.
Speaker A
And it's cool because it will prepare a full literature review and analysis for you. It'll also check evidence coverage, give you an analysis on what is the consensus here, what does the community think on particular questions. When you
Speaker A
run something like a deep search, it can go through literally thousands or millions of retrieved sources and finds the specific eligible ones and then limits it to the top one. And they even have this library system built in too
Speaker A
where you're able to sync all of your Zotterero files like I was showing you.
Speaker A
It really helps you work with the knowledge that you're already building. And I've been really impressed with how easy it is to connect to agents. If you want to learn more about consensus, I have two videos, how to use it as a
Speaker A
research management system and another on how to connect Zotterero and Obsidian. You can try Consensus for free using the link in my description.
Speaker A
So, what I'm most excited about today is that we're able to take that discovery capability and give it to a tool like the researcher agent. So, let's get it set up. So, if we go to capabilities here, and then over to the right, we can
Speaker A
see that we have MCP. And right now, I don't have any MCP servers connected.
Speaker A
So, what I'm going to do here is under the researcher profile, I'm going to go to import, click add to MCP, and it will create the connection to consensus. And the authentication is going to be through their authentication. So now I
Speaker A
can click authenticate and that should take me directly to consensus where I can now directly connect to Hermes agent. So I'll click connect and now we can see that consensus is active.
Speaker A
Perfect. Okay. So now that it's connected, why don't we test it to make sure that we have an actual MCP connection. Cool. So we can see it's now running MCP consensus search. It specifically ran for agentic memory poisoning, which is what we want to work
Speaker A
on. Great. And we can see the connection test succeeded. The consensus search for agentic memory poisoning returned 20 papers. That means that the Hermes researcher was now able to connect to that external consensus MCP server that we just set up. This is just a really
Speaker A
great way for you to be able to connect your existing agent to a platform that specializes in finding quality results.
Speaker A
But what I'm most excited for is a combination, not just getting academic research, but then also being able to take the current reality of what's actually being done with the academic research and combine them to get a balanced result. Okay, so we've taken
Speaker A
this same researcher identity, the same one that we built as a basic version in the first video, and now we've upgraded it where we've given it the LLM wiki research skills, so it understands how to communicate with the librarian agent.
Speaker A
We've given it its ability to read my existing Zotterero library to pull in all of my existing academic research.
Speaker A
We've now enabled it to go find new academic research and connect to it, all while having grounded citations in the types of reports that it will be able to produce. So each one has its own layer to it that all connects together into
Speaker A
the researcher agent which will then feed into our research team to improve the quality of the research we're doing.
Speaker A
So now that we have everything set up, why don't we actually run a test and see how it can work as part of a group chat.
Speaker A
So I'm going to create a new group chat for this and I call this skills-based research team. And what we can do is we can tag each of the people in this group chat. So now I'm going to message the
Speaker A
orchestrator and ask it to delegate to the researcher for a combined report that specific type of path we talked about to check zotterero and then also look for academic path using consensus.
Speaker A
I wanted to compare and contrast the two of them together. Then it will also check zotterero to see what we already have and then save a markdown report on agentic AI safety and give it back to the orchestrator who can then review and
Speaker A
quality check the report and present it to me for review. So this is the same process that I used last time where I message the orchestrator and tell it to delegate to the researcher before I pass it down to the librarian. So let's see
Speaker A
how it goes. Okay, so we can see that the researcher went through and it did find actually a bunch of reports. These are a lot more relevant research papers that I had added into Zotterero before.
Speaker A
So this did a better job than just asking it to look for poison because I asked it to search for memory and then limit the results to paper that harm memory. So it's able to find more papers. Okay, there we go. So that took
Speaker A
about 5 minutes and we can see that the researcher messaged the orchestrator and said, "Hey, I completed the combined reality and academic consensus briefing.
Speaker A
Here is the report. Here's the recommendation. Please review before passing to the user." That's exactly what we want. So we can see they are messaging each other back and forth.
Speaker A
Says, "The report is now under an independent evidence focused quality check. I'll return the acceptance decisions once that completes." Okay. Okay, so just while the orchestrator goes and checks that, let's take a look at the report. So we can see
Speaker A
that this specifically is a current reality and academic pass. The research question was, how should a systems be structured to reduce risk for memory poisoning? If you're not sure about what memory is, I have two videos that go
Speaker A
deeper into it. So I have my trusted LLM wiki, which you can see here, where something only makes it into my existing Wikipedia if I trust it. Memory can be more automated here through something like Nimosin or hindsight or the
Speaker A
built-in memory. So what's cool is we have all of these citations. So if we go down to the bottom, we can see specifically all the sources that were brought in here, including Microsoft, different operational safety mechanisms, different academic papers, and these are
Speaker A
all grounded throughout the system here. So that's using the grounded citation skill. And we can see here we have the academic pass, which is the consensus discovery. It went through and found some counter evidence and uncertainty, some unresolved gap. Oh, and here's the
Speaker A
zotterero that it brought in. So this is pretty solid report. I'd have to go through this more in depth, but it looks like it brought in quite a lot of information and pointed me at some high-val sources that I could then go
Speaker A
through and verify myself. It's in a very helpful format and human readable. But if we want to make it even more effective for agents, we can introduce different tooling that will help the orchestrator process this information better and also enable the librarian to
Speaker A
validate this. This is where we get into my existing knowledge management system, my librarian that I make available to my members. So in the next video, we'll bring in the orchestrator and the researcher into this system here and
Speaker A
then show how we can use even more code and tooling to connect them more effectively instead of just relying on an LLM. We can consider this the advanced research team. Now let's see what the orchestrator thinks.
Speaker A
Okay, great. So we can see that the orchestrator has now come back and it said that everything was high quality.
Speaker A
There were just a couple provenence correction revisions where it wanted a little bit more details on what it pulled from consensus and a little bit more detail on what it pulled from Zotterero. So, while I agree with this,
Speaker A
in the interest of time, I'm going to tell the orchestrator that this is actually good enough for now. I approve it. Please run the LLM wiki review skill.
Speaker A
Great. So, the librarian now has messaged me and said the review gated ingest is running. So, the librarian is now going to take this report and connect it to the rest of the wiki. This is using the built-in review skill that
Speaker A
I talk about more in my agentic librarian video and it's a companion skill to the LLM wiki built-in skill inside of Hermes that brings a human into the loop. Okay, there we go. So, we can see that the librarian has finished
Speaker A
its proposal. So, this is the report here. We can see it's under status of needs review. It has the proposed content waiting for my approval including a new concept called agentic memory poisoning that connects to other entries inside of my wiki like agentic
Speaker A
memory. So, the quality of this looks good. We can see it's citing the raw report that we just brought in from the researcher. The key takeaways of this report on how to prevent agentic memory poisoning is effectively to introduce a
Speaker A
control layer into the system so that you control what makes it into the trusted knowledge base. So I can include this report in the free Patreon article linked in the top comment if you're interested. And this is also why I have
Speaker A
this video on a trusted LLM wiki specifically to combat something like agentic memory poisoning. So it's nice to see that the research that I'm doing here corroborates the current system that I already have. And I also really like that it includes open questions
Speaker A
which effectively provides some research paths that I can go down in the future to continue expanding on this report. So this is where not only does the researcher help me find credible information, the librarian also helps me improve the quality of my research. So
Speaker A
we get this really nice synergistic workflow and now we can approve, reject, revise or defer. So now the librarian is going to connect that report on agentic memory poisoning. Great. There we go. So we just saw that agentic memory
Speaker A
poisoning popped in and it was able to connect to multi-agent workflows LLM wiki and agentic memory. So it brought in the exact concept that it said it cites the research report that we created and it connects to other
Speaker A
elements already inside of the LLM wiki. And then we get the debrief from the librarian saying we've now connected this to the index. Everything has passed the checks. So I hope that this shows you how improving the quality of the
Speaker A
researcher can increase the rest of the team. I'm happy with the results and I'm excited to see how this continues building out my LLM wiki with even higher quality research as I continue to improve the rest of my team. So, we've
Speaker A
taken that same basic researcher and we've upgraded it. We've given it more access to capabilities with a better research flow, access to existing research in Zotterero and being able to pull quality sources from a tool like consensus. All of this gets brought into
Speaker A
the team which gives the team more credibility and quality and more to work with while also giving me transparency on where the sources are coming from and how they all fit together.
Speaker A
You may have noticed that some of this was a bit manual. I did have to move things forward. What we can do is we can introduce something like a goal that allows the researcher to run on a loop
Speaker A
or schedule it so that it can run once a week and update the papers it found and the orchestrator can connect to that and monitor it for quality and for changes.
Speaker A
The next step for me is to improve the efficiency and the quality of the handoffs in the checks between the different agents. I want it to be more consistent so it's a reusable workflow that I can trigger and be confident that
Speaker A
everything is going to work perfectly. So that's what we'll get into in the next video with the Hermes canban feature which is a way that we can scale this type of workflow between the different agents in a much more
Speaker A
consistent and reproducible way. And we'll do this by upgrading the orchestrator agent and giving it access to more tools. If you found this video helpful, please like, hype, and subscribe as I really appreciate your support. If you're looking for more ways
Speaker A
to support me, please consider joining my YouTube and my Patreon memberships. I give my members access to insights, tips, and kits on knowledge management and AI workflows, including the upgraded researcher and some other features that I'm working on to tie my different
Speaker A
agents together. Thank you very much because you continue enabling me to make free videos like this one. Thanks again for watching and I will see you in the next video.
Topics:Hermesbot modeAI agentsresearcher agentMCPZoteroConsensusagentic memory poisoningLLM wikimulti-agent workflow
