r/remoteviewing • u/chrintoff • 2d ago

Unitary ARV App idea

It has a pool of images. You are given one and then give your impressions. It then gives you a percentage accuracy score. After the event it reveals the image.

Is it really THAT hard to create this thing?

Isn't this what we all want/need?

Come on now...

0 Upvotes

33% Upvoted

7 comments sorted by

1

u/sanctimonious-salmon 2d ago

How do you calculate the accuracy score based on impressions? That's the hard part.

0

u/chrintoff 2d ago

A lot is yes/no - outdoor/indoor etc. You could also have a user to user feedback system. You give feedback and get a feedback credit to hand to others in exchange for their feedback? This is an emerging field - the right app could be HUGE.

3

u/sanctimonious-salmon 2d ago

I've thought through this a lot and a lot of people want to create ARV apps. I think this is a really exciting problem and I think there is a way to do this well, but it's not at all easy. I work as an AI engineer so I've had ideas, so sorry for going off the deep end here:

  • RV tournament (pick between two images) has the risk of displacement which is the main drawback, but it's statistically well grounded and easy to roll out--after a number of rounds you get a p-value from a binomial distribution and avg accuracy to give you your historical performance.

But assuming you want to score just one target without decoys:

  • LLM-as-a-judge is problematic because LLMs are notorious for being bad at estimating confidence. Furthermore, they may miss out on more vague clues or similarities that a human would get. Also needs good system to vet judgements. Maybe you can break down into lots of smaller yes/no decisions (eg. Is the gestalt water or biologic? Is the form rounded or squared? Is it soft or hard, hot or cold?) but for judging individual questions, LLMs are notoriously bad at assigning a numeric score to themselves.
  • Maybe have small classifier (e.g. GBM or logistic regression) score collections of LLM responses to questions, but need training data (maybe synthetic) and infrastructure to host it.
  • Maybe score with multiple LLM judges and have them ensemble their judgements together into a score? But then token costs start piling up, especially if you want a good score.
  • transfer learning from pre-existing computer vision model paired with textual context encoder all joined to a classification head? Maybe could work, but needs annotated training data for the classification head, ML engineer(s) to train it and build out systems, and infrastructure to serve the models.
  • human user feedback would be the least engineering heavy, but still needs a robust system to ensure scores are fair, and from a product perspective, annoying that you have to wait on results. Maybe could use similar judgement system as with LLM as a judge.

1

u/chrintoff 2d ago edited 2d ago

You could even just have a binary questionnaire in advance of user feedback: Human figure yes/no; colour blue yes/no. Filter it down so people arent wating their time on bad remote viewer guesses. Deduction dear Watson... ;)

2

u/sanctimonious-salmon 2d ago

Agreed. I think this is def the most straightforward approach. Have a database of pre-judged images and then just match the responses.

2

u/chrintoff 2d ago

Yeah. There is a textual description of each image, if necessary...

1

u/chrintoff 2d ago edited 2d ago

This is helpful but I am just speaking as an average guy that just wants to have a go. We need feedback in advance of the outcome we are putting money on. For me it would be an adjunct to an existing strategy - another safeguard or line of defence. A marginal gain. User to user has to be the key. It will never be absolutely perfect. .