Auto-Pilot Disengaged – August 8, 2011

[TraderD]

Adam,

This is at the source of the statistical reliability problem. The 20 DMF was in confirmed shorting mode during all of September 2008 and had its first market direction signal change with a cover your short/buy signal on October 10, 2008. The robot did manage the trade well because it was looking for short positions durring all of that time Currently, the 20 DMF is already in unprecedented oversold territory and waiting for a long signal. Under such circumstances, the robot decides independently of the 20 DMF, and based on ST/LT statistics, whether to go long or short. But because the database only has a very few ST/LT combinations similar to today's, this is the exception to the rule that should be applied instead.
Billy

Billy, I'm copying my recent comment to the analogy made by Ivan Hoff on another thread:

I'm somewhat puzzled by the analogy. The robot reportedly cruised fine through the Fall of 2008 and didn't have to go to cash every night. Why is there a manual override now? Is this possibly a symptom of an over-optimized trading system (relative to available backtest/analysis historical data)?
Trader D

The rather simple-minded observation I have (admittedly, in complete hindsight) is that if the robot didn't include the ETF confirmation criterion (in its specific incarnation), we would be now in a rather impressive short swing trade and AFAICT a manual override into cash wouldn't happen, is that correct? If that is indeed the case, wouldn't you be inclined to think that the ETF confirmation is an over-optimized component of the robot? Would it make sense to go back and reduce the number of parameters tweaked in the robot setup to achieve greater consistency at the likely expense of lower overall return?

Trader D

[Billy]

Billy,
The rather simple-minded observation I have (admittedly, in complete hindsight) is that if the robot didn't include the ETF confirmation criterion (in its specific incarnation), we would be now in a rather impressive short swing trade and AFAICT a manual override into cash wouldn't happen, is that correct? If that is indeed the case, wouldn't you be inclined to think that the ETF confirmation is an over-optimized component of the robot? Would it make sense to go back and reduce the number of parameters tweaked in the robot setup to achieve greater consistency at the likely expense of lower overall return?

Trader D

It is true that not including the confirmation would have made a killing in the recent series of trade, but would have significantly deteriorated the robot performance in previous series of trades. It is only in hindsight that Pascal could make the discovery that confirmation improved performance during POMO periods but hindered it out of POMO periods. This information is obviously now seriously considered for inclusion in the robot rules and very likely will be in the future.

A manual override in cash would have happened anyway for new entries and the robot would already have covered its initial short position.

Now, let me kindly notice that if we change the rules after each losing trade, the robot will lose its objectivity and utility. Pascal is running new mixes of rules in backtests 24/24 with the latest observations mixes since a few days now to make sure of optimal necessary rules adaptations.

Billy

[adam ali]

Thanks, Billy, for the great feedback.

I'm no statistician but I do wonder at what point we can truly test a sufficient number of scenarios that we feel comfortable dipping our toe in the water again. Without sounding melodramatic, I think it's fair for me to be concerned that a similar situation to what just occurred (i.e., a 20DMF signal overridden with the outcome being a series of Robot trades that were stopped out), could happen again. In other words, not knowing what I don't know is an uncomfortable position to be in, and I don't know what other series of events could result in the EV system encountering another unknown, so to speak.

In hindsight, I wish I had raised the issue of the 20DMF moving below the 0% line forcefully. However, would that truly have changed anything in terms of your and Pascal's response? I'm not trying to be cheeky, here; I'm merely trying to ascertain whether poking and prodding on my (and everyone's) part is helpful to the goal of improving the system.

It's a difficult problem: we want to trade with statistical probabilities on our side, but at the same time, do we have a sufficiently robust sample set to truly feel comfortable we "have the bases covered"?

[Billy]

It's a difficult problem: we want to trade with statistical probabilities on our side, but at the same time, do we have a sufficiently robust sample set to truly feel comfortable we "have the bases covered"?

Adam, you want HIGH probabilities on your side. None can be found in any direction based on Friday's close however you look at it. I've probably read more than 100 blogs and newsletters this weekend and whatever any indicator they use, they are all amazed how they are in "terra incognita".
I doubt any algorithm from anyone could provide a valid high probability trading plan for more than a few minutes currently.
Cash is King! Cash is a position with a stop once we see high probability trades again.
Billy

[TraderD]

...It is only in hindsight that Pascal could make the discovery that confirmation improved performance during POMO periods but hindered it out of POMO periods. This information is obviously now seriously considered for inclusion in the robot rules and very likely will be in the future...
Billy

Is the "POMO periods / ETF Confirmation" a case of correlation or a case of causation in the process of developing new robot rules (http://stats.org/faq_vs.htm)? I would feel quite more comfortable with the ETF confirmation rule if the rationale for its use during POMO periods was inferred based on logical grounds prior to a data mining exercise that shows the superior performance it produces.

Trader D

[Billy]

Is the "POMO periods / ETF Confirmation" a case of correlation or a case of causation in the process of developing new robot rules (http://stats.org/faq_vs.htm)? I would feel quite more comfortable with the ETF confirmation rule if the rationale for its use during POMO periods was inferred based on logical grounds prior to a data mining exercise that shows the superior performance it produces.

Trader D

Unfortunately, your link is dead so I'm not sure I understand your question well.
Pascal is the leader in mathematical/statistical expertise and is surely better qualified than me for responding.
Billy

[TraderD]

Unfortunately, your link is dead so I'm not sure I understand your question well.
Pascal is the leader in mathematical/statistical expertise and is surely better qualified than me for responding.
Billy

Sorry, Billy, the link should be devoid of the question mark: http://stats.org/faq_vs.htm

HTH,

Trader D

[Pascal]

Is the "POMO periods / ETF Confirmation" a case of correlation or a case of causation in the process of developing new robot rules (http://stats.org/faq_vs.htm)? I would feel quite more comfortable with the ETF confirmation rule if the rationale for its use during POMO periods was inferred based on logical grounds prior to a data mining exercise that shows the superior performance it produces.

Trader D

I introduced a confirmed short signal rule when I noticed at some point that large players were selling SDS while the 20DMF was issuing a short signal. I thus concluded that the short might be a false signal and overruled it at that time (early POMO days). It is on that base that I included the SDS confirmation in teh 20DMF rules - that was later on changed by the four inversed ETFs confirmation.

The important issue is not regarding the 20DMF. I think that the issue is to render the robot strong enough to sustain minimal damage in case of market direction error or even to overrule the 20DMF at some point. The solution I believe is in better sorting out the weak and strong robot trading signals. I am working on that. Another item is to revert to cash whenever the LT/ST signals are so completely apart - which indicates that the model has reached its programmed statistical limits. We are in such an exceptional situation right now.


Pascal

[Timothy Clontz]

Billy, Pascal,

Do I understand correctly that in the future the Robot will have occasional cash positions?

Tim

[Billy]

Billy, Pascal,

Do I understand correctly that in the future the Robot will have occasional cash positions?

Tim

Yes Tim,
This could be a possible new rule at times. We'll know after full backtesting of new suggested rules have been done.
Billy