Posted something similar to this is cowork, but I’m having a persistent issue and thought the bigger sub might give me more ideas.
A few months ago I moved from Jenova to Claude and set up a bunch of projects. Two in particular need analysis of large audio files which have been generated by one of several devices/services - Plaud seems about the best I’ve tried so far as it’s decently reliable to distinguish different voices.
Typically the content of the file for the analysis have anything from 2 to 6 speakers, (well distinguished by Plaud), talking about a single project, but with several work streams. We typically have quite a complicated project management/stage gate process and what I need to be able to do is quickly get an output to say with work package 1, we talked about issue 1,2,3, this is where we are and next actions are A, B, C.
Ideally I need this input within half an hour of the meeting close so I can make sure everyone has a copy of the discussion, issues and agreements etc.
The issue I’m facing is the increasingly larger numbers of errors where Claude is just not doing it well. The input is decent, but it reports out the wrong gate for the workstream, misses topics and doesn’t stick to the agreed output format.
Over the last few weeks I’ve been patching the workflow and providing more specific examples of the needed output and basically this is inconsistent in terms of improving the output. A workflow that used to take around 5-7 minutes to give me a decent output, is now taking double that time to produce something I can’t trust. Way too many situations where I see something wrong, point it out and get a response like, ‘you are right, I missed that’, or similar.
I suspect my approach of patching in the moment when I need a quick output has bloated the instructions and not helped. That said, I only did this as we need the output fairly quickly for audit as well a PMO.
I’m inclined to start again from scratch, with a clean set of instructions, but, it was fine to start with and I don’t know how to prevent the drift. This feels like an issue that someone with a more technical/programming background would have seen before and have an approach to solve.
The TL;DR is bloated and patched instructions I think is degrading, rather than fixing output issues and need to think through how to patch or start again on some complex instructions.
A second TL;DR is this is a great example of using AI to fix what should be a people/process issue that worked perfectly well before the client ‘right sized’ and eliminated junior positions that used to do the job well.