Large Language Model features in your app in minutes

An introduction to PlayFetch

If you’d like to join the waitlist for access to PlayFetch you can do so here. If you’d like to get in touch with us directly please send us an email to hello at playfetch dot ai. You can also watch a very similar walkthrough here on YouTube

Large language models (LLMs) are incredibly powerful. They’re enabling features and products that we could previously only dream of and have changed the way we think about adding intelligence to our applications. But as we discovered while we were building Dreambooks they also require a completely new way of working.

Tasks that ought to be simple are anything but. Iterating on a prompt or chain with other members of your team, developing and testing new versions and determining whether they outperform your previous ones, and integrating the feature into your app in a manner consistent with other APIs are all much more challenging than you’d expect.

The problems largely stem from the non-deterministic nature of the responses you get from these models. Where in traditional software development we might write unit tests to verify behaviour and know we’ll get the same result each time, here we write prompts which result in a different output every time. This flexibility is the reason these models are so valuable, but it makes developing with them really challenging. Plain text APIs are not great for software reliability!

PlayFetch brings back some of the certainty you’re used to, while also allowing you to iterate and improve on your features in a structured way while sharing the responsibility of this development with other members of your product team. We’re seeing product managers, content strategists, researchers, and dedicated prompt engineers all directly involved in the development process alongside traditional software developers, and this shift means we need new tools.

To demonstrate some of the features of PlayFetch let’s walk through the process of developing a new feature in a notes app we’re working on. The list view for my notes always looks like a complete mess, and I’d love it if instead of the first few words from the note, I could automatically generate useful titles for each note and use those instead.

So let’s create a new project in PlayFetch and start by writing out the first version of our prompt. We want to generate a title for a note so we’re going to want to pass that text in as a variable. We can do this by either selecting the text we wish to replace and clicking create input or by simply enclosing the text in double curly braces like this {{note}}. I can also select which model I’d like to run my prompt with there. Let’s pick Claude Instant.

Getting started: writing our initial prompt in PlayFetch, adding our note input variable, and selecting the model we want to use.

Now that we’ve done that we can click over to the test data tab and add a few example notes for us to use while we iterate on our prompt. Let’s add one about badgers, one about AI, and one about chocolate. Now I can run the prompt three times in parallel to get my first responses which will appear on the right.

Simple testing: adding test notes and getting multiple outputs simultaneously.

These titles look a little bit long and some of them are inside quotes while others are not. Let’s correct those things by changing our prompt a little. We can see the new responses on the right (much better) and if I switch back to the Prompt versions tab I can see how PlayFetch stores the history of my iteration as well as highlights the differences between each version. I can also label, annotate, comment and more, but that’s a topic for another article.

Getting it right: iterating on our prompt - note the two versions on the left hand side along with the green highlight changes in version 2.

Now all that’s left to do is publish this as a RESTful JSON endpoint that I can hit from my notes app. To do this I just need to click into the Endpoints screen and click add endpoint.

Publishing an endpoint: here we’re exposing our prompt as a RESTful JSON endpoint that we can access from our notes app.

And that’s it! Now I have an endpoint that when given a “note” parameter will return me a title. If I decide I want to update this feature (perhaps I want to support multiple languages?) I can iterate on and test my new version from within PlayFetch and then publish it all without having to rebuild my application.

If you want to try PlayFetch for yourself please reach out to us at hello at playfetch dot ai or join our waitlist here.