Interactive Visualizations in Joomla Using R

In the last five years, data visualization has moved beyond paper-ready plots and graphs to interactive visualizations that are easy and inexpensive to generate. Incorporating interactive visualizations into web sites requires more work and higher authority levels than just including a graphic, but it can be well worth the effort for some articles. The article that follows describes how to address the security and logistical issues involved in using interactive graphics on a web site. The article is divided into the following sections:

• An Example WebGL Visualization
• WebGL and Other Interactive Visualizations are Implemented in JavaScript
• Configuring JCE to Allow Pasting JavaScript Visualizations into Articles
• Configuring PHP to Increase Timeouts for Saving Very Large Articles
• Configuring Server-side Includes
• Minify HTML Files
• Gzip HTML Files
• Using Schema.org Structured Data for Interactive Graphics Search Engine Optimization (SEO)
• Other Useful Embedded JavaScript Graphics
• Google Trends
• ggvis is intended to be used within Shiny or RStudio
• Interactive heatmaps with d3heatmap
• Scatterplots with scatter3js
• Interactive linecharts with dygraph
• Spherical Network Diagram with threejs
• Network diagram using htmlwidgets
• Line charts using googleVis
• Notes

An Example WebGL Visualization

Before describing how to embed interactive visualizations in a Joomla article, it is probably worth showing an example of the type of visualization that is described in the article. The visualization below shows a constrained two-product optimization problem where maximum profit is at the top of the dome. The vertical planes are constraints on maximum unit sales; when the price is too low, sales exceed production capability. The visualization was prepared in R using the rgl package for WebGL/OpenGL accelerated graphics. On desktop computers you can rotate and flip the visualization, but on most mobile devices this won’t work, as they do not have the graphics rendering hardware needed for WebGL/OpenGL.

Charts generated with rgl can be included in a Joomla article using the <iframe> method; the Sourcerer server-side include method does not work well for this package.

WebGL and Other Interactive Visualizations are Implemented in JavaScript

Most if not all interactive visualizations are implemented using a combination of HTML and JavaScript, and the files can be quite large; it is not unusual for a visualization file to be 1 or 2 megabytes. This presents two problems for putting these visualizations in a Joomla (or WordPress) article:

• Because JavaScript can be used maliciously, the default Joomla editor strips out JavaScript and HTML whenever an article is saved, so that an unknowing or malicious blog comment or author’s article won’t put a cross site scripting vulnerability into your site. The default editor in WordPress appears to allow JavaScript.
• Because the files are quite large, when you paste it into the Joomla (or WordPress) editor, it will probably time-out before it finishes saving the article. Saving an article with multiple visualizations will probably not work at all.

The sections that follow describe how to install the JCE editor and configure it to allow only authorized users to put HTML, JavaScript and PHP into their articles, and then how to configure PHP so that it won’t time out when you save a large article with visualizations. A subsequent section goes one better and describes how to set up server-side includes so that you can include multiple visualizations in one article.

Configuring JCE to Allow Pasting JavaScript Visualizations into Articles

The default TinyMCE editor in Joomla strips out HTML and JavaScript. To give you better security control over HTML, PHP, and JavaScript, install the JCE (Joomla Content Editor) extension and configure it so that it compresses JavaScript, and CSS and then enable editing JavaScript, HTML, and PHP–but only for administrators or other user groups that you trust. You will need to go through this step even if you set up server-side includes. Do not authorize JavaScript for the default profile:

• Figure 1 shows using the Joomla Configuration panel to change the default site editor to JCE.
• Figure 2 shows the Component->JCE Editor->Global Configuration panel where you enable compression.
• Figure 3 shows the JCE Profile->Administrator panel where you enable Javascript, HTML (CSS) and PHP editing

Configuring PHP to Increase Timeouts for Saving Very Large Articles

Because the HTML and JavaScript for interactive graphics is quite large, you will need to increase the max_execution_time and max_input_time PHP configuration values so that PHP won’t time out while you are saving an article. If you configure server-side includes, you do not necessarily need to do this step. Where to change these values will depend upon how your web hosting firm has set things up. You may be able to change values in .htaccess or in a local php.ini file–but you may have to put one in every single directory on your site. If you have Cpanel/Web Hosting Manager (WHM) access, this is by far the easiest solution. For many users, the following steps will work:

• Figure 4 shows finding the PHP info tab in the Joomla Configuration panel.
• Figure 5 shows looking up the location of the php.ini file in the Joomla Configuration panel
• Figure 6 shows the values needed for a custom php.ini file
• Figure 7 shows the Cpanel/Web Hosting Manager panel for modifying the PHP time-out values

Configuring Server-side Includes

The previous discussion shows how to set up PHP to handle pasting large files into an article; this is manageable for articles with a single interactive graphic, but is inconvenient at best, and it just does not work for articles with more than one visualization–the time required to save the article is just too long and trouble prone. Setting up server-side includes is a much more convenient way to handle multiple visualizations. To do this, you will need a Joomla extension that allows in-line embedding of PHP code. Sourcerer is one package that does this using the syntax shown in Figure 8. This is by far the easiest way to include interactive visualizations on your site. In this case, I created a directory for my interactive visualization files and then used scp to copy the files from my desktop to the web hosting server.

You will need to modify the settings for uploading media files or get shell access and use scp; by default, Joomla prevents uploading HTML and JavaScript files for the same security reasons that it does not allow including them in articles by default. If you have multiple users on your site, you should get the PRO version of Sourcerer so that you can disable the inclusion of PHP for default user profiles.

Minify HTML Files

Because the .html and .js files for interactive images are very large, you will want to minify them before you upload them to your web server so that your minify/compress extension does not have to work as hard. There are a number of tools to do this. Yui-compressor is probably the easiest to install and use, but is by no means the best at this point. There are some Perl libraries that work better, but you will need to write some code to use them. For the command below to work, you will need to create a new directory for your minified files.

For the interactive graphics in this article, only the googleVis had any benefit from this.

Gzip HTML Files

Gzip is by far the best thing to improve performance with interactive graphics. Prior to implementing gzip, Google Page speed would not get though processing this page; with gzip, it gets a score of 68 which is great considering the size of the files. Although you can (and should) turn on the gzip/deflate modules in Apache, it is better to go ahead and gzip the interactive graphics files since they are static anyhow.

Using Schema.org Structured Data for Interactive Graphics Interactive Graphics Search Engine Optimization (SEO)

Because the Google and Bing web crawlers don’t see text that is contained in JavaScript, and because these JavaScript visualizations will not appear in your image sitemap, you will need to add information to tell the search engines that the visualization exists and what it contains. To do this, select an appropriate datatype from schema.org data types, and include markup in your article. Figure 9 is an example of this type of structured data markup.

Once you have marked up the article, test the structured data for errors using Google's Structured Data Testing Tool. Google does not currently implement all of the structured data tags, but it clearly uses structured data in determining how to display search results. The Google query “bruce moore software services event educational” will bring up a list of events listed in the Selected Speeches and Presentations on this web site; another example of how this markup will help search display can be found in the query “arts district chorale events” will present a similar display.

Embedded JavaScript Graphics within an <iframe>

Once you begin to use JavaScript on your site, there are a number of additional charts and visualizations that are easy to include; the sections that follow describe some of these, and give examples of some additional R visualizations. Many of these visualizations are generated to be stand-alone visualization files and are poorly behaved if you bring them in as a server-side include. For these, you will need to use an <iframe> tag. Because this tag can be used to replace an entire browser window allowing a malicious article or comment author to impersonate your site, many sites restrict the use, but it can allow you to incorporate some visualization files that were intended to be stand-alone.

All of the R package visualizations are copied or derived from the package examples.

Google Trends

Google Trends is an easy way to get an understanding of the seasonality of some of the queries on your web site. This Google Trends query history for Dallas Cowboys, New York Jets, New York Giants, and New England Patriots shows strong seasonality in searches for NFL football teams.

In some circumstances it is useful to embed a Google Trends chart in your article, as shown in Figure 10. The </> symbol in the lower right corner of the Google Trends chart will give you the JavaScript needed to embed the chart in your article. The Google Trends code is snippet is JavaScript but it generates an <iframe>; it is probably implemented this way to get around the prohibition on direct inclusion of <iframe> in many content editors.

ggvis is intended to be used within Shiny or RStudio

The R packages ggvis is designed to be run under either RStudio or Shiny, but it can export HTML that can be included in an <iframe> as shown in the markup in Figure 11. Figure 12 shows the R code used to create the visualization shown in Figure 13. Getting this to render outside of RStudio and Shiny is still a work in progress.

require(ggvis)
require(knitr)
#outfile <- ggvis(mtcars,~mpg, ~wt, fill=~cyl) %>%
#  layer_smooths(span = input_slider(0.5, 1, value = 1)) %>%
#  layer_points(size := input_slider(100, 1000, value = 100)) %>%
#  view_static() %>% htmltools::html_print(viewer = NULL)
#system(paste("cp ",outfile,"images/interactive/dfw_R_meetup_UseR_2015_ggvis.html"))
plotGGvis <- ggvis(mtcars,~mpg, ~wt, fill=~cyl) %>%
  layer_smooths(span = input_slider(0.5, 1, value = 1)) %>%
  layer_points(size := input_slider(100, 1000, value = 100))
plotGGvis

## Warning: Can't output dynamic/interactive ggvis plots in a knitr document.
## Generating a static (non-dynamic, non-interactive) version of the plot.

outLines <- knit_print.ggvis(plotGGvis)

## Warning: Can't output dynamic/interactive ggvis plots in a knitr document.
## Generating a static (non-dynamic, non-interactive) version of the plot.

writeLines(as.character(outLines), "images/interactive/dfw_R_meetup_UseR_2015_ggvis.html")
#export_png(chartGGVis
#           ,file="images/figures/dfw_R_meetup_UseR_2015_ggvis.png")

Interactive Heatmaps with d3heatmap

Heatmaps are a popular and useful visualization. Figure 14 shows the R code for the d3heatmap package used to generate the heatmap visualization in Figure 15, where you can mouse over individual tiles to see the specific data represented in the tile–a useful but not particularly flashy interactive feature. The markup for including the visualization in a Joomla article is similar to the markup shown in Figure 11.

Charts generated with d3heatmap can be included in a Joomla article using the <iframe> method; the Sourcerer server-side include method does not work well for this package.

require(d3heatmap)
url <- "http://datasets.flowingdata.com/ppg2008.csv"
nba_players <- read.csv(url, row.names = 1)
chartD3 <- d3heatmap(nba_players, scale = "column")
print(chartD3,file="images/interactive/dfw_R_meetup_UseR_2015_d3heatmap.html")
#print(chartD3)

Interactive Scatterplots with scatterplot3js

Scatterplots are probably the bread-and-butter visualization for all statistics, but it can be frustrating to try to trace back a particular plot point to a data element in the data set; you can see an outlier easily in the plot, but finding the primary key value in the dataset may take some time. The interactive features of the scatterplot3js visualization generated with the code in Figure 16 and the shown in Figure 17 make identifying outliers and understanding data much easier.

Charts generated with scatterplot3js can be included in a Joomla article using the <iframe> method; the Sourcerer server-side include method does not work well for this package.

require(threejs)

## Loading required package: threejs

chart3js <- scatterplot3js(mtcars$wt,mtcars$cyl,mtcars$mpg,renderer="webgl")
print(chart3js,file="images/interactive/dfw_R_meetup_UseR_2015_scatterplot3js.html")

dygraph

Line charts are a standard type of chart but adding interactivity with mouse-over highlighting for detail data makes them more useful. Figure 18 shows dygraph code that generates the interactive linechart shown in Figure 19

Charts generated with dygraph can be included in a Joomla article using the <iframe> method; the Sourcerer server-side include method does not work well for this package.

require(dygraphs)
lungDeaths <- cbind(mdeaths, fdeaths)
chartDy <- dygraph(lungDeaths)
print(chartDy,file="images/interactive/dfw_R_meetup_UseR_2015_dygraphs.html")

Spherical Network Diagram using threejs

Network diagrams are useful for understanding complex datasets and map data. The threejs package provides a way to generate interactive network diagrams that can be manipulated in a web browser. Figure 20 shows the code used to generate the network visualization shown in Figure 21.

Charts generated with threejs can be included in a Joomla article using the <iframe> method; the Sourcerer server-side include method does not work well for this package.

Network diagrams can help to understand complex data;

require(threejs)
data(flights)
dest   <- factor(sprintf("%.2f:%.2f",flights[,3], flights[,4]))
freq <- sort(table(dest), decreasing=TRUE)
frequent_destinations <- names(freq)[1:10]
idx <- dest %in% frequent_destinations
frequent_flights <- flights[idx, ]
ll <- unique(frequent_flights[,3:4])
chartGl <- globejs(lat=ll[,1], long=ll[,2], arcs=frequent_flights,
        arcsHeight=0.3, arcsLwd=2, arcsColor="#ffff00", arcsOpacity=0.15,
        atmosphere=TRUE, color="#00aaff", pointsize=0.5,renderer="webgl")
print(chartGl,file="images/interactive/dfw_R_meetup_UseR_2015_threejs.html")

Network Diagram Using htmlwidgets

Network diagrams are useful for understanding complex datasets. The htmlwidgets package provides a way to generate interactive network diagrams that can be manipulated in a web browser. Figure 22 shows the code used to generate the network visualization shown in Figure 23.

Charts generated with htmlwidgets can be included in a Joomla article using the <iframe> method; the Sourcerer server-side include method does not work well for this package.

require(htmlwidgets)
library(networkD3)
data(MisLinks, MisNodes)
chartWid <- forceNetwork(Links = MisLinks, Nodes = MisNodes, Source = "source",
             Target = "target", Value = "value", NodeID = "name",
             Group = "group", opacity = 0.4)
saveWidget(chartWid,"images/interactive/dfw_R_meetup_UseR_2015_htmlwidgets.html")

## Error in normalizePath(basepath, "/", TRUE): path[1]="images/interactive": No such file or directory

Line Chart using GoogleVis

The Google Visualization APIs provide powerful tools for web visualizations; the googleVis package provides a way to take advantage of these tools from within R. Figure 24 shows the code used to generate the interactive line chart shown in Figure 25.

Charts generated with googleVis can be included in a Joomla article using both the Sourcerer server-side include method or the <iframe> method.

require(googleVis)
chart <- googleVis::gvisLineChart(data=mtcars,xvar="mpg",yvar="wt")
print(chart,file="images/interactive/dfw_R_meetup_UseR_2015_googleVis.html")

Notes

This article was written in RStudio using the knitr package.