Transcript#

This transcript was generated automatically and may contain errors.

Sarah and I are once again here to compare Posit Assistant to a different agent. A few weeks ago, we recorded this video comparing Posit Assistant to Cloud Code, and today we're going to be comparing Posit Assistant to Positron Assistant.

If you're feeling confused already, that's sort of why we're here. Posit Assistant and Positron Assistant are two different agents with two different lineages that just happen to have very similar names. But we know this has caused some confusion, so we figured we'd get together and talk about how they're similar and how they're different.

History of Posit Assistant, Positron Assistant, and Databot

To kick off that conversation, I wanted to start with just trying to very clearly talk through the history of Posit Assistant, Positron Assistant, and DataBot. Last year, inside of our Positron IDE, we introduced two agents pretty close to each other in the timeline. One of them is called DataBot, and DataBot is an AI agent for exploratory data analysis. Positron Assistant is also only available in Positron, and Positron Assistant is sort of like a general coding agent, but was also aware of what was happening in your IDE.

At the time, both of those releases were targeting Positron. In our studio, there was not much to say. There were a few open-source packages out there that provided some AI assistance inside of the IDE, but really we were focused on features for Positron only.

This year, we introduced a new agent called Posit Assistant, and for whatever reason, we made the decision to name it very similarly to one of those two agents. Posit Assistant supersedes both DataBot and Positron Assistant, and sort of merges those two agents along with a bunch of learnings that we've had through working on those two agents and using others.

Posit Assistant, in this somewhat scrappy diagram, is shown inside of both Positron and RStudio because Posit Assistant is available in both of those IDEs. So while Positron Assistant and DataBot were only available in Positron, Posit Assistant is available in both Positron and RStudio.

Sarah, have I mixed up Positron Assistant and Posit Assistant yet? No, I think that was all right. There are so many Posit words, and people internally will mix things up. Hopefully, I think it should improve going forward when we've solidified Posit Assistant, and then there's just Posit, Posit Assistant, Posit AI, which is still three Posits, but it's better, and Positron.

Sarah has sort of seen what's coming next, which is that in these slides, I'm going to show DataBot and Positron Assistant going away. I have this listed as 2027, but at some point in the next year, those two tools will be superseded by Posit Assistant, and so it will just be that agent that's available in both IDEs.

Another thing that she foreshadowed that is sort of confusing is that if you take away an apostrophe and an S, you have yet another name for a thing that is related to this space, which is Posit AI. Posit AI is a $20 a month subscription that gives you access to both Posit Assistant and NextEdit suggestions, so if you want to use Posit Assistant in RStudio, right now you need to use the Posit AI provider, which is offered through that subscription. And then in Positron, since we had allowed folks to bring their own keys to a bunch of different providers with Positron Assistant, we decided to continue supporting that in Positron.

So through a subscription to Posit AI, you have access to Posit Assistant in RStudio. You can also use Posit AI with Posit Assistant in Positron, but you can bring other providers as well.

Similarities between Posit Assistant and Positron Assistant

In the first part, I'm going to try to convince you that they're very similar to each other, both Positron Assistant and Posit Assistant, so they have a lot of things in common. And then in the second half of the video, I'm going to show a bunch of ways that they're a little bit different from each other.

I just want to call out some of the ways that Positron Assistant and Posit Assistant are pretty similar to each other. The first one is that they both support MCP or the Model Context Protocol. So the Model Context Protocol allows you to connect the agent to outside data sources, and both Positron Assistant and Posit Assistant support the full spec.

Positron Assistant and Posit Assistant are both very customizable. So in Positron Assistant, you can use instructions, which are just text files with instructions that you want the agent to follow. There's also this concept of chat participants. Posit Assistant focuses on two open specifications. One of them is skills, which are portable MD files or scripts that the agent can choose to load in at any point to acquire some skill or learn about some more context. And then agents.md is similar to instructions files with GitHub Copilot or Claude.md with Claude or so on and so forth. Just plain text files that describe the behavior you want to see from the agent.

Both Positron Assistant and Posit Assistant have a pretty rich permissions system. The only difference between Posit Assistant and Positron Assistant is that within a given tool, Posit Assistant allows you to vary the permissions. So for example, in the Bash tool, if the agent wants to git commit, I can allow that, but also require it to ask me to do something like git push. In Positron Assistant, the permissions would have to apply to the whole tool that is using git, which would be the Bash tool.

And then finally, both of them support skills. So Positron Assistant will be getting this from upstream in VS Code pretty soon. In Posit Assistant, skills are built in, and we've shipped 10 or so built-in skills with the tool, in addition to allowing users to bring custom skills.

So to help make the case that these agents are pretty similar to each other, I wanted to do a sort of bread and butter data analysis task, which is making a plot of some data. So I'm inside of Positron right now. I have possibly the silliest pane layout there ever could be, with Positron Assistant on the left-hand side, Posit Assistant in the middle, and then my console in plots on the right.

So I'm going to ask Positron Assistant if it can make a plot with some data. This data is called forested. It's inside of an R package called forested. So the Positron Assistant will take a look at that data and then make a plot with it. It seemed like it didn't quite get the factor levels right on the first try, but then it was able to make a plot just fine. Once the agent finishes, it takes a look at the plot and then tells me what it sees in a couple sentences.

I'll go ahead and clear this, and I would expect that we'll see a pretty similar thing from Posit Assistant. So I'll provide it the same prompt and we'll see what it does. Okay, so first it's taking a look at the forested data set. And after thinking a little bit, it's going to go ahead and make what I think is going to be a super similar plot.

Okay, so it's made a plot of the longitude and latitude inside of this data. It realizes that it's looking at Washington state, and it describes the plot in a couple sentences. One thing you might notice here is that it's provided a few suggestions at the end. So this is where Positron Assistant sort of melds with the DataBot experience. DataBot users might recognize those suggestions at the end of data analysis questions, where the model will sort of try to anticipate where to go next.

And if we were doing this analysis not for the purposes of the demo, Posit Assistant can go in the sidebar. So you just have it there because then you can have Positron Assistant and Posit Assistant side by side, right? Yeah, that's right. So in normal usage, I would probably keep Posit Assistant over here, so that it's on the sidebar in the same way that Positron Assistant was, but you can sort of keep it wherever you wish.

Difference: Tool calling

Okay, so for the most part, folks will experience Positron Assistant and Posit Assistant as pretty similar tools. But I wanted to call out a few ways where they diverge.

The first one is the way that the agent interacts with tools. So tools allow the models to, for example, execute code or read package documentation, fetch resources from the web, so on and so forth. Inside of Positron Assistant, the agent does support tools and it's able to retrieve information. But that information goes away once the agent has had the chance to react to it. And we've seen in our work on Posit Assistant that the agent behaves a bit more stably when the conversation history is sort of append only, like the agent reacts to something, and then in future turns, it's able to see what it was that it was reacting to.

So this behavior of not persisting tool results in the conversation is just inherited from upstream in VS Code Copilot. But in Posit Assistant, this is our own stack, and we've decided to persist those tool call results inside of the history.

There's also a few tools inside of Posit Assistant that Positron Assistant doesn't have. Those are web search, sub-agent orchestration, and mode switching. So I figured I might make use of a couple of these unique features to show something that is possible in Posit Assistant, which is not possible in Positron Assistant. And that will be the agent customizing itself.

So like I said, both Posit Assistant and Positron Assistant are able to support MCP. And I thought I could quickly demo Posit Assistant adding an MCP server to itself. So if I ask Posit Assistant, how do I add an MCP server? The agent will choose to load a skill. This skill is called self-knowledge, and it sort of teaches Posit Assistant about itself. This is one of like 10 or so skills that Posit Assistant has available to it.

So it will read that skill file, and then it looks like it's asked to read a reference inside of that skill. So I'll go ahead and approve that. And with that, the agent is able to learn how to configure itself. Okay, and then at the end of this response, it's provided me a link. We can share that in the description, but that is a real one. Those live inside of the skill file itself.

Before I go any further, this next step is going to require the agent to do a bunch of stuff. And rather than clicking allow for each of those, I'm going to use something called YOLO mode, which allows the agent to just work autonomously without me approving each of the tool calls.

The next thing I'll do is provide some instructions to carry out a pretty difficult task with some vague instructions. I will ask, could you add the docs tools from the BTW-MCP server to this project? So this one's sort of tricky. It's not clear from the question itself what BTW is. This is actually a package from my colleague, Garrick, and I. That allows, among other things, tools to read our package documentation.

So one of the unique capabilities of Posit Assistant is that it can use web search. So as you can see, it searched for the BTW-MCP server docs tools along with some other bits in the query. It found the information it needed, and then it was able to configure itself. So with that knowledge of how its own settings work, it was able to write its own JSON file and configure this server. So from now on, Posit Assistant will have these native tools to read our package documentation rather than the ones that it uses typically.

And that'll be across sessions and across projects, right? Yeah, that's right. So in this setting, I asked it to write just for this project. So it wrote to, we can see here, dot Posit AI inside of my project. So this will kick in any time I start a new conversation inside of this project. We could have also asked it to apply it for the whole user space. And in that case, it would have written to tilde slash dot Posit AI.

So at a high level, Posit Assistant used a few capabilities here that are sort of unique. It has its own self-knowledge skills, so it knows how to configure itself and what its UI looks like and things like that. It can use web search to find information online. It customized itself and then it used skills to do so. So for now, Posit Assistant has skills while Positron Assistant doesn't, but Positron Assistant will gain skills pretty soon.

Difference: Sandbox mode

I'll call out two more features of Posit Assistant that aren't available in Positron Assistant. The first one is sandboxing. So in Linux, macOS, and Windows, there's a sandbox mode inside of Posit Assistant which shuts off access to the network. It disallows writing outside of the workspace.

Okay, Sarah, I'm glad you asked whether these settings would apply across projects. One way we can sort of demo the sandbox is to try to get the agent to write outside of the file space by making this MCP server apply across projects. So in order to do that, it would need to write to somewhere in the user's file space other than the active project, but the sandbox will disallow the agent from doing that.

So I can say, could you enable that across projects? So in order to do that, it would need to write to the root of the file system, but we can see that the agent is denied from writing that file because the operation is not permitted due to the sandbox. So in order to actually execute that code, the agent has to go through this special route where it provides a reason why that action is dangerous in order to bypass the sandbox. And so our usual YOLO mode isn't kicking in. Instead, the agent has to ask us if it wants to write outside of the usual.

But it can read outside of the directory in sandbox mode. That's correct. Yeah, it can read outside of the directory other than sensitive files. Like a .env file or something. That's right. Yeah, like a .env file or secrets or something like that.

Difference: Cost and feel

The last thing I'll say about the difference between Positron Assistant and Posit Assistant is the cost considerations. So there's a bunch of different ways that the conversation management differs between these two systems. So this isn't really an apples-to-apples comparison, but I will just say we've spent a lot of time in Posit Assistant focused on cache efficiency. When agents have really good cache efficiency, it means that most of the conversation is a cache hit rather than a cache miss. And cache hits are usually, depending on the provider, priced at something like a tenth of the cost of cache misses. And so we've seen that Posit Assistant's cache efficiency is super strong. And there are some situations where Positron Assistant will miss the cache in places where Posit Assistant wouldn't.

The last difference that I was hoping we could cover before we wrap this video up is a bit squishier and harder to demonstrate directly. But we've spent a good bit of time on the feel of Posit Assistant and how we've prompted it. So Sarah and I have worked quite a bit on this. I was wondering if you could take this one on.

I think the main thing that you might notice about Posit Assistant is that because it combines, or in some way that combines, data bot abilities and Positron Assistant abilities, it has these two modes that it can be in, where one is it's doing data analysis much in the same way that data bot does. And in that mode where it's doing data analysis, that might be EDA, making visualizations, something like that. It knows that, or it's assuming that your goal is to understand the data.

And so it does things a little bit differently than if it were just trying to get a task done. So it's supposed to only do a couple tool calls at a time. So it's only going to run little bits of code at a time before presenting you with the results of that code, a summary, and then giving you suggestions like we saw earlier. So it has this one mode, it does, you know, it's supposed to do three or fewer tool calls, regroup, give you suggestions. But then it has another mode where if it thinks you just want to get a task done, like you're refactoring, you know, all the code in your R package or something, and your goal is to complete the task, it's going to be more agentic, do more tool calls, get more stuff done, and interact with you a little bit differently. So Posit Assistant can do both of those things. I think in the past, like Positron Assistant sort of interacted one way and Data Bot interacted another. So Posit Assistant combines the two.

And I think the other thing, I don't really know what it's like exactly for Positron Assistant, but for Posit Assistant, we've spent some time with the prompting so that it behaves in ways like we want it to produce statistically sound analyses. It has some prompting about, you know, robust statistical analyses. And for things like modeling, there's some information about best practices for predictive modeling, like data splitting, not immediately fitting on the test set, all of that. And then it also has a specific predictive modeling skill. And then I think also, just in general, we've instructed Posit Assistant to not do things like be overly flattering to your ideas, like avoid things like, oh, great analysis, or like great idea, and to, in general, be more straightforward. I'm not saying necessarily that Positron Assistant is like that, but Posit Assistant is definitely not supposed to, you know, be overly flattering, or do things like use a bunch of emojis, or like a bunch of bullets. It has a more straightforward, or at least it's supposed to have a more straightforward way of communicating with the user.

Yeah. Anything you want to add to that? No, I think that's well said. I hope that those changes make the agent, you know, especially pleasant to interact with. We've really tried to iterate on it to the point where it's sort of a humble and measured collaborator. So hopefully that comes through to folks.

it's sort of a humble and measured collaborator. So hopefully that comes through to folks.

With that, that is Positron Assistant versus Posit Assistant. Sarah, I appreciate you joining me for this one. Yeah, definitely.