Ok, here's my thoughts:
The difficulty with this experiment is trying to weed out confirmation bias and file drawer effect (i.e. forgetting that you had negative results, and focusing exclusively on the positive results)
Simply trying to "forget" past expierience actually goes against what LOA's basic principle is - that you are in fact responsible for everything that has (and will) come to you. Trying to forget or dismiss your previous experiences only implies that your missfortune or failed efforts were the cause of outside forces, and not your own control.
So, the first thing the subject has to do is claim complete responsibility in the fact that he/she has previously had bad luck in attaining lucidity (or achieving goals in lucids). One way to do this is sort of simple - "I failed previously because I have not yet tried this new method which I am about to try"
To combat this, we need a baseline for the experimentee and result sheets that reflect an unbiased result. And the experimentee should continue doing exactly what they have been doing thus far.
I think the scoring system needs to somehow be blind to the subject himself. By this I mean, the subject should not keep track of the "+" and "-" themselves, or at least should not be able to look back on previous results. One reason; the results themselves can have an effect on optimism of the experiment and the person can get sidetracked by it. If the subject notices a string of negative results, they may lose optimism and then interest. On the other hand, if there should appear a line of positive nights, the person will use this track record as an indicator that "it's working" and use this confidence in succeeding nights, rather than the LOA. Not that it's a bad thing, but the experiment IS about the effect of LOA in the end.
How does one keep track, but not really keep track? I dunno, it's kinda hard to do. I think one should agree to a long term commitment - like a month or two. Then, the subject submits the results to someone else who records the positive or negative result. The subject will at first keep track in their head for a while, but hopefully lose track and forget until the end of the experiment (where the entire course of the experiment can then be tallied up).
1) The results need to be comparitive. Apples to apples. If person X works towards a lot of dosh, and person Y works towards peace in the family... it's not comparable. Or is it?
I don't think the intention itself would be that relative. All "results" can be paired down into one universal variable: "positive" (or comforming with the person's intention), or "negative" (not conforming to the intention).
EXAMPLE: Person a wishes to have lucid wich involves someone who's passed away, person b simply wants more lucid dreams. Even though person a's report could come in varous degrees of success while person b is a simple "yes" or "no" - both can report if each night they had a positive or negative overall.
2) Because if you wish for something that isn't feasible, and you don't believe that it could happen, that may arguably influence results.
Ah, this is also another trapped way of thinking that is actually the opposite. See, the more outrageous or difficult the goal, the more convincing the LOA is when you start seeing results. Sure, you could sit there and be like "Ok, I'd like to be driving a 2008 silver convertible by next summer". As you know, LOA won't just suddenly pull the car up to your drive while you sit there and wait. It'll drop hints along the way and give you clues as to how you can get the car:
EXAMPLE:
The following week you might be out in your old clunker car (or riding your bike) and see the car in an ad, on the street. That's when you go

.
A month later you might receive some junkmail saying "Get this new Motorola phone with service and you will be entered into a drawing to win THIS car" - upon which you'll go

.
A week later, you'll "accidentally" drop your phone from your balcony and it will shatter into a hundred pieces. As you scoop up the pieces you'll run across the Motorola ad in the garbage (which you most likely threw away) and go

Anyway, I got off topic.
For the scoring system, there might be a way each person could record the results themselves without seeing previous results. I bet I could come up with a Flash app that you simply enter a value like "1" and "0" and then it writes it to a text file which you can access after 30 or 60 days. That might work.
Bookmarks