ggplot2: Don’t Try This With Excel

Learning R

Building on the original blog post “VBA to Split Data Range into Multiple Chart Series” by Jon Peltier, and the R version in this blog and in his blog, Charts & Graphs blog “shows 4 charts of the same data to demonstrate what Excel chart users are missing by not having a more powerful charting tool. /?/ These analytical displays are not readily available to even advanced Excel users”.

He is using base graphics to draw the plots, I will present the ggplot2 version.

View original post 479 more words

Game Theory

“Zero-player games are essentially ‘physics’ or any of the hard sciences – nature abides by its own axioms, not those Axioms we define as ‘economic rationality’.

One-player games can be thought of as optimization programs, where a single player is tuning a process to a goal he controls, with parameters he defines.

Two or more players form a game we restrict with the Axioms of Rationality. We require more than the “natural order” because we seek to describe movements through state-spaces defined by mathematical rules. Those state-space descriptions are given in the language of payouts weighted by probabilities – so-called “expectations” – as well as number of rounds and players.”

Have Faith in Science

I have a friend who loves linguistics.  Especially the counts of certain words, and their positions next to the counts of other certain words, and what the distances between those words and their counts implies about the topic of the sentence they are in. This relatively modern field is called lexigraphical density analysis, which is just fancy terminology for what I said earlier. My friend is particularly interested in scanning the Bible and creating reference counts, and trying to surmise from words counts what subjects there might be (the “topics”, from above).

There’s just one problem: he thinks he hates math.

The bigger problem is that I may have confused him when I said, “well, honestly, I hate ‘math’, too. But I love algebra, calculus, and statistics, and I’m even getting into abstract math and topological spaces.  But I hate [spirit fingers] ‘math’. To boot, I’m not very good at it.”

Well this just didn’t compute with him. He sincerely did not understand how I was getting a Master’s in Economics, or maintained a geek-level fascination with consuming and creating data graphics for web developers, or budget forecasts for the State.

I thought about this all weekend.  About his confusion, and my apparent duality – no, hypocrisy! – about my love and hate for so-called “mathematics.”

This morning it occurred to me – in one of those sort of quiet flashes you have on your own, where you exhale and realize you’re not insane and that it might be okay, after all – that this is exactly my relationship with a Christian God.  He is comprised of undeniably good things, but the name which encases these goals is mishandled by anyone and everyone who’s come into contact with Him.  In some cases, even those most familiar with mathematics and God have been the ones to increase the level of dogmatic requirement to “be a believer.”

Let’s forget the biggest hitch, here, by saying up-front that I think the things I refer to when I say “algebra, calculus, and statistics, and even abstract math and topological spaces” are God. I believe that science and faith are two halves of the same question we have been asking since ‘ask’ was a word – and probably many hundreds of thousands of years before that, too.  We ask questions with, of, and by language (and therefore history) in the case of faith.  We ask questions with, of, and by numbers (and therefore the future) in the case of science. The conversion process happens now, today, when these two horizons overlap.  (And yes, I do mean to imply that past science is a matter of faith.)

The schism of these two incompatible formats is not an implication that their answers are similarly incompatible. This is a formatting issue. And today, just like video and sound formats are played in perfect synchrony with one another, it’s possible for the first time to align the signals in faith and science.

But this process of alignment – steering, navigating, transcending – requires a feedback mechanism to account for error and miscalculation.  This is a founding principle of the scientific method, not the “undoing” of science itself – as many members of various faiths report when unexpected or contradictory results are borne from scientific pursuit.  Note that this puzzles scientists greatly – friction of information is the root cause of innovation and discovery to them, while for those of faith it seems a root cause for cognitive dissonance and revulsion.

When the front-edge of the forecast made by the Bible passes – if it has not already – God and mathematics will still be here.  The Earth will too, most likely.  All three of these sacred entities must be better understood in the context of the others, and no human institution should consider itself above the process of correcting errors that are not only glaring, but terminal. Lacking adaptive ability is a death sentence in cultural memetics as much as it is in human genetics.

But no matter who or what God turns out to be, we must continue to ask questions in both a mathematical and faith-based framework.  Just as each of these must have its own feedback mechanism for error correction, they are themselves the feedback mechanism for the other! This leaves us in the perilous condition of having two tools, each of which requires the other to calibrate – errors in that type of system compound quickly. Regardless of on which side of the equation we throw them or how arbitrarily or fervently we assert their ownership, the system requires a balance, an equilibration.

Science has come a long way in the last 500 years, and those of Faith rightly seek a balance. But they will not find it by assigning errors to the side of the equation replete with error-correction mechanisms – indeed, that reflexively arbitrary assignment of errors by the Catholic Church (“the Earth is flat”), and their conclusive disproof, has inspired much of the modern science we use today. Not dinosaur blood and straw-man missing links, but electricity and refrigeration, GPS and back surgery. Does anyone who owns a GPS think the world is flat, and what do Catholics today think of that fact? The prospect of refining our beliefs is one that we should aspire to, document, and help our children remember – we should not reject or rebuff such opportunities.

How Color Can Trick The Eye: 12 Fascinating Optical Illusions

Michael Sandberg's Data Visualization Blog

Source: Ann Swanson, 12 fascinating optical illusions show how color can trick the eye, The Washington Post, February 27, 2015,

It sounds inane, but the dress question was actually tricky: Some declared themselves firmly in the blue and black camp, only to have the dress appear white and gold when they looked back a few hours later.Wired had the best explanation of the science behind the dress’ shifting colors. When your brain tries to figure out what color something is, it essentially subtracts the lighting and background colors around it, or as the neuroscientist interviewed by Wired says, tries to “discount…

View original post 420 more words

How to Create Coefficient Plots in R the Easy Way

Didier Ruedin

Presenting regression analyses as figures (rather than tables) has many advantages, despite what some reviewers may think

coefplottables2graphs has useful examples including R code, but there’s a simpler way. There’s an R package for (almost) everything, and (of course) you’ll find one to produce coefficient plots. Actually there are several ones.

The one I end up using most is the coefplot function in the package arm. It handles most common models out of the box. For those it doesn’t, you can simply supply the coefficients. Here’s the code for the coefficient plot shown. The first two lines are just to get the data in case you’re interested in full replication.

The default in arm is to use a vertical layout, so coefplot(m1) works wonderfully. Often I prefer the horizontal layout, which is easily done with vertical=FALSE; I also add custom margins so that the variable…

View original post 361 more words

Crawling – Mining Twitter Data menggunakan R

menulis yang terpikir

Dalam aktivitas content mining, data mining, social network analysis dan sebagai bagian dari pekerjaan data science, maka melakukan mining terhadap media sosial merupakan hal yang ‘wajib’. Dalam entri blog kali ini saya akan menuliskan mengenai crawling percakapan dan konten dari media sosial Twitter menggunakan bahasa R. Penjelasan mengenai R ada di halaman wikipedia ini. R dibangun secara crowdsourcing dimana banyak saintis dan programmer membuat modul modul khusus untuk meningkatkan fungsi fungsi dari bahasa R.

Salah satu package / library / modul yang menarik adalah twitteR, modul ini dibuat untuk mengakses API dari Twitter, sehingga kita bisa melakukan operasi operasi seperti melihat profile, melihat daftar teman, daftar followers, pencarian kata kunci dan lain lainnya. Operasi yang sering saya lakukan adalah pencarian kata kunci untuk kemudian saya lakukan data mining, sentiment analysis atau social network analysis.

Langkah langkah yang perlu dilakukan adalah yang pertama kali…

View original post 204 more words