SWM Defence system stands for ‘Star War Mosquitoes Defence’ System.

under which ‘there is no spot at all where the mosquito is safe’ and ‘don’t stop there. You can adjust the system to destroy all irritating elements in your surroundings. For instance the neighbor’s cat. Or the neighbor’s house. Or undesirable planets.’  here is the wired article.

Here is the force behind this new invention: Intellectual Venture Lab, the pilot project, The Venture firm behind it, and seems it is expanding. HBS idea cast: re-inventing invention.

here is the high speed camera the project was using: up to 1,000,000 frame/sec !! here is the method used to detect mosquito: Particle Image Velocimetry.


February 19, 2010

let me tell you a little story:

there was once a young man who dreamed to reducing the world to pure logic. Because he was a very cleaver young man, he managed to do it. When he finished his work he stood back and admired it. It was beautiful, a world purged of imperfection and indeterminacy, countless acres of gleaming ice stretch to the horizon. So the cleaver young looked around the world he created and decided to explore it. He took one step forward and fell flat on his back. You see, he forgot about friction, the ice was smooth and level and stainless, but you couldn’t walk there. So the cleaver young man set down and wept bitter tears. As he grew into a wise old man, he started to understand that roughens and ambiguity aren’t imperfections, they are what make the world turns. He want to run and dance, on the words and things that scattered upon the ground, all battered and tarnished and ambiguous. The wise old man saw that is the way things were. But something in him still hope for the ice, where everything is radiant and absolute and relentless. Though he come to like the idea of rough ground, he could live there. So now he is trapped between earth and ice, at home of neither. And this is the cause of all his grieve.

wiki, movie.

summary and roadmap

February 18, 2010


  • microstructure/order book/agent based modeling/market maker/double auction/gaming/liquidity-> largestly heuristic but more pragmatic
  • dynamic system approach/econophysics, distribution, complexity, much on theoretical level
  • hardware/latency/routing/optimal execution, institution dominates with fancy word such as ‘dark pool’
  • HMM-Kalman filter/BDNetwork ML(EM etc.)/Classifier (VM, NN, TREE, BOOSTING etc) ->advanced technical analysis, advanced classifier
  • RLearning, GA, evolutionary market hypothesis -> capture market dynamics(?), prediction power(?)
  • pair trading, Stats Arb, –> technically simple yet basic, BREAD AND BUTTER OF algo trading.
  • last, most basic and fundamental is money management and risk management. control of order flow, system monitoring.

information theory address both problem, estimation(model)/inference(observation) and optimal betting/money management(kelly/drawdown/portfolio) together on the fundamental level. MPT address the second problem for multivariate case.

let start with most basic and most fundamental proble: stat arb

  • modelling dynamics of stat arb [kalman filter application, with time dependency, long winding?]-> paper
  • stat arb in US equity market -> paper
  • pair trading, with EM algo, model mean reversion with state space model, [clear and succinct, kalman for estimation, EM, smooth KF for model validation, interesting idea is using Ornstein-Uhlenbeck stop time as exit strategy]05-> paper
  • flexible least square for data mining and stat arb apply on ETF, imperial math/bluecrest, Tesfatsion’s FLS page -> paper [ this paper discussed the implementation of an online  FLS algo, and online SVD for priciple component]
  • data mining for algorithmic asset management on Futures, Montana 09 -> paper
  • FLS and stat arb in layman’s term ->[the is a waste of time, so much so for B-school professors]  paper
  • Review of Statistical Arbitrage, Cointegration, and Multivariate Ornstein-Uhlenbeck [clearly written, intuitive, my favorite so far.], Attilio Meucci BB alpha head-> link
  • Engle Granger, erro correction representation and cointegral -> paper
  • Engle Granger 87 -> paper did this one won them nobel? repost again for emphasis.
  • UCDavis Econ time series course with all the reference and list of topics -> website
  • pair trading, performance of relative arbitrage rule [utility sector, average crossing, bid-ask bounce, trading cost, question left open is: a) how to choose the threshold? b) within trading period till when before next calibration you should not trade? ]-> paper, 06 IFC working paper
  • here is an paper describe all SA strategy [wast of time] -> paper
  • an intelligent SA system [a good outline of strategy, (moving window to estimate B, and NN-Garch for interval estimation and corresponding trading strategy]-> paper
  • high frequency pairs trading with u.s. treasury securities[touched implementation issue e.g. repo financing cost in rates market ]-> paper
  • statistical analysis of cointegral vector(strucuture change) -> paper
  • THIS google search would yield interesting result, example here, here ( Alexandre 08), here(alexandra investment related to AP Alexandra?)here, here cointegration alpha: index tracking(carol alex 02), here (carol alex 03), here( carol alex JPM 05), here( this one came from SMU), optimal hedge cointegration (carol alex)|| here(fractional cointegration ”09), copper market efficiency, metal future efficiency
  • an VERY good tutorial: NYU Courant Stat Arb course, let’s call it Dyna arb? the author Farshid Maghami Asl is a VP in GS FICC.  other courant people(robert almgre, algo 08, marco avellaneda present some course presentation here), there is a good portion dedicated to vol trading and stoch vol estimation using KF, but not enough details is given out. Chpt 14 start to touch some interesting general topics as financial system, robust control, bayesian optimal control and control of balance sheet, again just touched, not enough details.
  • pair trading, quantitative method and analysis -> book
  • pair trading, capture profit and hedge risk with SA strategy -> book
  • Tsay’s course website for Time series analysis -> 1 2

Fallacies; Psychological manipulation(abuse, bully, narcissism) ; Propaganda;

Bayesian inference;

  • pdf v.s. likihood function
  • MLE, estimation
  • estimation bias
  • BI and court case: Howland will Forgery, Sally Clark, Lucia de B.
  • Bayesian search. USS Scorpion P(here|nofind)=(P(nofind|here)*P(here)) ../(P(nofind|here)*P(here)+P(nofind|nohere)*P(nohere))
  • clearly there is a strong relationship between Kalman filter and HMM, both of which belongs to Dynamic Beyesian Network, here is a tutorial on this topic. Mackey’s toturial on Beyesian network. Murphy K presentation on DBN, and his ’02 thesis. Murphy K’s publication page.  Micheal Jordan’s publication and tutorials. Judea Pearl, Bayesian causality (Judea asking very fundamental and interesting questions here) his publication. this explains why RMF is interested by physicist while DMN is interested by AI community. reasoning(prediction, estimation, classification) is causal, describe nature law is non directional(parameter and variable interchangible, observational(algebra explains observation non-experiment v.s. interventional, control experiment, algebra explains observation + causality lead to prediction(?)).P(r|w) v.s. P(r|do(w)). Simpons paradox and solution; reverse regression (gender, salary, qualification). aka adjustment problem, covariate selection problem. wiki on Belief Propagation just learned that Judea Pearl is father of Denial Pearl.
  • K. Murphy tutorial on: RL here, Graphical model and BN here


February 14, 2010

rule 1. there is always a reason if there is a profit.

rule 2. knowledge and skill is abundant (this and this when you can Google it), cutting edge knowledge & skill (means you cannot Google it) is rare, judgement/capital/organization skill is precious.

rule 3. whatever you are doing, imagine a million people are doing the same and doing better, keep on refining and researching until you can no longer find any other people sharing your idea, then you are out of mediocracy.

technology arsenal

February 14, 2010

  1. python tutorial
  2. max dama’s python system monitoring, IB via python
  3. connect IB with MATLAB post and this and this
  4. API: Lime (people at lime); IB (retail shop); Wolverine Ex (check their software package brokerage turn technology company? out source IT).
  5. the competition and information dissemination on this topic seems extremely fast on the website
  6. algo podcast, algo forum, algo open platform, algo human resource, algo evaluation, Suite – Alib/Aladin(!!&Suite people), algo Event processing blog, Algo empirical research
  7. and you have to check this resume and think about what you want to do here.
  8. this is what i’m looking for: how to front run iceberg order! and this, dama post on dark pool/smart routing
  9. related to above is GS’ thoughts on Electronic Trading
  10. two trading problems from dama’s class
  11. some DEShaw insights
  12. Kelly post on Dama, and this, and this, and this, information theory betting post,  trader’s ruin, trading objective  postthrop 07 on Kelly; simulation and optimization post; system optimization post; hill climb optimization; human crude optimization
  13. open source system development, including marketc
  14. a long and interesting post, about a whole system implemented in R, with tree bagging(random forest) and kelly etc.
  15. another system based on svm idea implemented in M, background and this and this and this decision tree post
  16. the very first system description and implementation, and this and this, commercial system design
  17. Dama’s SVM post, BL post; Bayersian learning video, dirichlet process post; something about statistical learning; Information theory post; SVM application to trading post; base AI and trading post; SVM basic application; SVM basics; SVM paper
  18. data preprocessing post; over fitting and bias 1, 2, 3; vol v.s. leverage post, and a case study; some data to play with, clarifi a capital IQ tool for portfolio management.
  19. Dama’s hands-on Matlab class, LDA; having fun with matlab, and this. this paper contain a literature review of attempted empirical trading strategy and history of pair trading.
  20. Rapid Miner SVM post, Rapid miner intro here.
  21. market regime indicatior

long memory

February 10, 2010

here is a website dedicated to long memory issue in various domain of research interest.  interesting articles includes under others, fractional, multifractionality. D Farmer a main contributor. an good illustration of HMM from model to metamodel: Rama Cont wrote a very readable overview of time series analysis here,  here is someone recently updated on this issue (interesting point maybe now calendar effect weakened in recent year). he coauthored with J.P. Bauchaud who run Capital Fund Management (largest french hedge fund). here is a ’93 interview on CFM with founder J.P. Anguliar, and ’09 story on his death, moral of the story: in long term gliding will kill you. CFM’s research archive here, particularly the random matrix theory in ’99 and ’07. see also this one. French risk manager has a distinctive taste for risk see this coppula in finance, correlation is only matters in extreme negative value, therefore the conventional mean-variance method is inadequate, see matlab toolbox.

work of Andrei Leonidov: market mill dependence, non-gaussian dependant pattern, long memory, non-Markovian nature.

well, work of benoit mandelbrot, and his yale website.

Ian Kaplan (his guy worked in Prediction for a while) digged out and digested Hurst’s original paper here. quote – ‘when comes to wavelet, i’m the guy with a hammer to whom every problem is a nail’

Dynamics approach, Joint PDF approach:

1) robust structure without predictability

2) the variation of certain speculative price

3) conditional probability as a measure of volatility clustering

4) price clustering and discreteness:is there chaos behind noise

5) conditional dynamics driving financial market

A Dias: using copula model short term dynamics in high frequency.

dependent structure paper

there is this whole fund industry which focused on fee, and there is this whole prop trading shops preaching ‘make money with us’ and rack up margins see this.