Bayes' Theorem in Technology

by Dax Bradley

The Bayesian model for statistical analysis is highly popular in settings that require predictive strength to succeed.  Industries are numerous, from academic areas such as astronomy, to industrial applications to include telecommunications and pharmaceutical companies (May, 2014).  In the telecom sector, the method has been employed for some time.  Akamai Technologies in Cambridge uses Bayesian methods to better understand network latencies, internet traffic, and to identify clusters of users (Galkowski, 2014).

The computational complexity of the Bayesian model would suggest that primarily technology-based efforts would embrace this form of analysis (Edelsbrunner, 2014).  Google’s robotic cars, and Microsoft’s own early spam filters stood on the strength of Bayes theorem (McGrayne, 2011).  Google’s strength lies not only in the high level of technical prowess, which is obvious, but also by a factor not apparent: the large amount of data to draw from.

The Bayes approach relies heavily on what is known already to make a successful prediction on what is likely to happen (Russell & Norvig, 2010).  Google translate uses Bayes Theorem in the translation services, by drawing on a large pool of translation data already completed by humans (King, 2015).  So, rather than undergoing large amounts of translation algorithms, the software instead draws from experience, samples the data, and predicts the highest certainty.  The method has been applied by Google not only in translation and mapping, but also in controlling driverless cars. 

The tech giant claims that their autonomous vehicles have logged as many as 140,000 miles along the Pacific Coastal highway, all without incident (Matyszczyk, 2011).

It is easy to see how Bayesian methods are highly prized in the technology sector.  The approach is also critical in health technology assessment.  While no formal definition of Bayesian tools in healthcare technology has been established, proponents believe that it provides for conclusions in a form most appropriate for decisions specific to patients and decisions affecting policy.  That is based upon the following conclusions (Spiegelhalter, Myles, & Jones, 1999, para. 4):

• Data are interpreted from a study in the light of external evidence and judgement
• The form in which conclusions are drawn naturally contributes to decisions
• Prior plausibility of hypotheses is taken into account
• Skepticism about large treatment effects are formally expressed and cautiously interpreted
• Use of Bayesian methods in healthcare technology assessment are to be pursued carefully; guidelines, software, and critically evaluated case studies are needed

References

Edelsbrunner, P. (2014, November 17). Bayesian statistics: What is it and why do we need it? [Blog post]. Retrieved from JEPS Bulletin: http://blog.efpsa.org/2014/11/17/bayesian-statistics-what-is-it-and-why-do-we-need-it-2/

Galkowski, J. (2014, October 17). How companies use Bayesian methods [Online forum comment]. Retrieved from http://andrewgelman.com/2014/10/17/companies-use-bayesian-methods/

King, M. (2015, July 20). Bayes’ theorem and what we do [Blog post]. Retrieved from https://www.whitehatsec.com/blog/bayes-theorem-and-what-we-do/

Matyszczyk, C. (2011, March 4). Google’s self-driving car goes all Dale Earnhardt. Cnet. Retrieved from https://www.cnet.com/news/googles-self-driving-car-goes-all-dale-earnhardt/

May, J. (2014, October 17). How do companies use Bayesian methods? [Online forum comment]. Retrieved from http://andrewgelman.com/2014/10/17/companies-use-bayesian-methods/

McGrayne, S. (2011). Why Bayes rules: The history of a formula that drives modern life. Retrieved January 19, 2017, from https://www.scientificamerican.com/article/why-bayes-rules/

Russell, S., & Norvig, P. (2010). Artificial Intelligence: A modern approach (3rd ed.). Boston, MA: Pearson.

Spiegelhalter, D., Myles, J., & Jones, D. (1999, August 21). An introduction to bayesian methods in health technology assessment. The BMJ, 319(1), 7208. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1116393/

Bayesian Model Challenges

By Dax Bradley

Possible disadvantages to Bayesian learning methods can be significant.  For one thing, the information gleaned from the effort could be theoretically infeasible.  The act of specifying a prior datum is challenging.  Generally, it is necessary to specify a real number for every setting in the real-world parameters (Langford, 2005).  One potential solution is to acquire languages that allow more compact specification of priors.  The task of learning a new language may be prohibitive.  Another solution is to simply lie.  In other words, create a specification that is not accurate.  As deceitful as this sounds, it is not meant that way; in fact, it has been shown to be effective (Langford, 2005).

Another known disadvantage to Bayesian modeling lies in the analysis of causality.  There is a great challenge in analyzing the conditional probability links between various nodes.  Even with physical causal models and known variables, the computational burden is great, since the task equates to inverting a finite-element model (Sankararaman, 2015).  This is especially daunting if the analysis revolves around an acyclic model.  The assumption of independence among variables implies no latent cofounding variable in Bayesian modeling; however, often there are acyclic latent cofounding variables which can force the model to be seriously biased (Shimizu & Bollen, 2014).

Prior choice problems present challenges in the Bayesian methodology.  Another scenario in which researchers using the model is in astronomy.  Scientists have attempted to predict the number of extra-solar planets in this way.  Improper priors are generally disallowed here, so other strategies have been suggested, such as Intrinsic Bayes, Fractional Bayes, and Expected posterior factors (Jeffreys, 2006, p. 11).  The potential defeat of using such methods is that data may be inadvertently duplicated, so allowances must be made.

Many forms of industry employ the Bayesian model of statistical probability.  In drug development, the approach is widely regarded for its accuracy and natural interpretation.  It does, however, demand no small degree of planning.  The flexibility of this method creates an environment prone to potential computational errors.  Without proper planning, the various outcomes of intense computation may be unreliable, prompting some teams to opt instead for a frequentist model (Gupta, 2012).

References

Gupta, S. (2012, June). Use of Bayesian statistics in drug development: Advantages and challenges. International Journal of Applied & Basic Medical Research, 2(1), 3-6. http://dx.doi.org/Use of Bayesian statistics in drug development: Advantages and challenges

Jeffreys, W. (2006). Current challenges in Bayesian model choice: Comments [Lecture notes]. Retrieved from Department of Statistics University of Vermont: http://astrostatistics.psu.edu/scma4/Jefferys.pdf

Langford, J. (2005, April 23). Advantages and disadvantages to Bayesian learning [Discussion group comment]. Retrieved from http://hunch.net/?p=65
Sankararaman, S. (2015). What are the limitations of Bayesian Networks? Retrieved January 18, 2017, from https://www.quora.com/What-are-the-limitations-of-Bayesian-Networks

Shimizu, S., & Bollen, K. (2014, August). Bayesian Estimation of Causal Direction in Acyclic Structural Equation models with individual-specific confounder variables and non-Gaussian distributions. Journal of Machine Learning Research, 15(1), 2629-2652. Retrieved from http://jmlr.org/papers/volume15/shimizu14a/shimizu14a.pdf

Bayesian Theorem - Advantages

by Dax Bradley

Creating complex models from complex data can prove to be an arduous task, even in a system of computers.  Using even existing data, a computing system must perform many calculations, often at the expense of vast resources.  It can be difficult to express a problem mathematically, particularly using highly details models available through direct or indirect complex mechanisms.  It would be no small challenge for a computing tool to interpolate shapes and behavior from raw video footage, providing a prohibitively large amount of variability and noise (Hertzmann, 2004, p. 5).

Bayesian reasoning provides a methodology for many data-modeling problems.  By searching for logic in uncertainty, Bayesian methods provide a unified approach that on the surface is like human logic.  Bayesian logic provides three main areas of interest:

1. Principled modeling of uncertainty

2. General purpose models for unstructured data

3. Effective algorithms for data fitting and analysis under uncertainty

Courtesy: CBS

There are advantages to Bayes learning method.  One area is in interpolation.  When faced with a challenge in engineering, there is always a balancing act to determine whether the time and resources a human would consume vs. a computing system.  With the formulation of an engineering system, a world model must be built, along with a “controller” for that environment.  Bayesian methods interpolate this to the extreme because the Bayesian prior can be a delta function on one model of the world (Barak, 2005).

The Bayesian method also obeys the likelihood principle.  In a case where two proportional likelihood functions come from two distinct samples for , then all inferences about both groups will be similar (Jones & Huddleson, 2015).  It also provides interpretable answers in a convenient setting for a wide range of models.  This includes hierarchical models as well as missing-data problems.  Less training data is also required; in fact, lower training data is preferred with the Bayes naïve approach (Chen,  2011, para. 3).  A naïve Bayes classifier will converge more efficiently than discriminative models such as logistic regression.

References

Barak, B. (2005). Advantages and disadvantages of Bayesian learning. Retrieved January 17, 2017, from http://hunch.net/?p=65

Chen, E. ( 2011, April 27). Choosing a machine learning classifier [Blog post]. Retrieved from http://blog.echen.me/2011/04/27/choosing-a-machine-learning-classifier

Hertzmann, A. (2004). Introduction to Bayesian learning [Lecture notes]. Retrieved from https://www.dgp.toronto.edu/~hertzman/ibl2004/notes.pdf

Jones, A., & Huddleson, E. (2015). Bayesian analysis: Advantages and disadvantages. Retrieved January 17, 2017, from https://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_introbayes_sect006.htm