2025-10-16T16:36:10+00:00 http://dwdii.github.io/ Daniel Dittenhafer atom@dittenhafer.net 2018-05-08T00:00:00+00:00 http://dwdii.github.io/2018/05/08/Delivering-ActiveMQ-Messages-To-ElasticSearch <h2 id="delivering-activemq-messages-to-elasticsearch-with-logstash">Delivering ActiveMQ Messages to ElasticSearch with LogStash</h2> <p>I was faced with a need to share messages sent through <a href="http://activemq.apache.org/">Apache ActiveMQ</a> (actually <a href="https://www.redhat.com/en/technologies/jboss-middleware/amq">RedHat JBoss AMQ</a>) to <a href="https://www.elastic.co/products/elasticsearch">ElasticSearch</a>. While evaluating options, using <a href="https://www.elastic.co/products/logstash">LogStash</a> seemed like an obvious approach which needed a closer look. It wasn’t immediately apparent how one might connect ActiveMQ to ElasticSearch with LogStash. There are several input plugins for LogStash which look promising, but what are their differences? Pros/cons? This blog post chronicles some of the research so you or my future self can refer to it when needing to integrate these technologies.</p> 2016-10-23T00:00:00+00:00 http://dwdii.github.io/2016/10/23/Digit-Recognition-with-Bootstrap-Aggregating <h2 id="digit-recognition-neural-network-with-bagging">Digit Recognition Neural Network with Bagging</h2> <p>For a recent assignment in the Machine Learning and Big Data course of my Master’s program, I was asked to add bagging, or <a href="https://en.wikipedia.org/wiki/Bootstrap_aggregating">bootstrap aggregating</a>, to a digit recognition neural network. We were provided an <a href="https://github.com/zatonovo/deep_learning_ex/blob/master/digit_recognition/ex_mnist.py">example digit recognition network written in Python with Keras and Theano</a>, and asked to enhance it to improve on the the default 0.984 accuracy.</p> 2016-09-09T00:00:00+00:00 http://dwdii.github.io/2016/09/09/One-Account-or-Two-Customer-Service-and-Social-Media <h2 id="one-account-or-two---customer-service-and-social-media">One Account or Two - Customer Service and Social Media</h2> <p>Working in the airline industry, airline related analytics are naturally of interest to me. I bumped in a slideshare presentation from SimpliFlying, an airline brand strategy firm, regarding various airline social media successes. The Delta Airlines example caught my eye. Delta has a special twitter account, <a href="https://twitter.com/deltaassist">@DeltaAssist</a>, dedicated to listening and responding to customers. The goal metric mentioned is customer service, though it is not clear how they measure it. I’m curious if they use any text analytics for either detecting issues to assist with, or for measuring their customer service performance (sentiment). It certainly seems like a good fit.</p> 2016-08-13T00:00:00+00:00 http://dwdii.github.io/2016/08/13/Garuda.Data-Apache-Phoenix-For-.NET-Developers <h2 id="garudadata---apache-phoenix-for-net-developers">Garuda.Data - Apache Phoenix for .NET Developers</h2> <p>On June 20, 2016, Microsoft released a preview of their <a href="https://www.nuget.org/packages/Microsoft.Phoenix.Client/">Microsoft.Phoenix.Client</a> on Nuget.org. This package provides a .NET framework compatible collection of classes to interface with the <a href="https://phoenix.apache.org/">Apache Phoenix Query Server</a> for <a href="https://hbase.apache.org/">Apache HBase</a>. I had been evaluating Phoenix and HBase in the prior weeks and the release of the .NET client library was very interesting to me. It was the only .NET compatible client I was aware of and I immediately began experimenting with it.</p> 2015-12-27T00:00:00+00:00 http://dwdii.github.io/2015/12/27/CCNY-Graduate-Student-Symposium <h2 id="ccny-graduate-student-symposium-2015">CCNY Graduate Student Symposium 2015</h2> <p>Last spring I had the opportunity to present my project <em>U.S. Births and Unemployment Rate 2007-2012</em> at the City College of New York’s Graduate Student Symposium. When the call for projects was originally issued, I dismissed it rather quickly mainly due to the distance and travel needed for me to attend. The symposium is in New York City, while I live on the Space Coast in Florida. It just didn’t seem feasible to fly up to NYC for 1 day, so why bother going through the application process.</p> 2015-01-27T00:00:00+00:00 http://dwdii.github.io/2015/01/27/US-Births-And-Unemployment-2007-2012 <h2 id="us-births-and-unemployment-rate-2007-2012">U.S. Births and Unemployment Rate 2007-2012</h2> <p>In this post I’m continuing the theme of sharing my work in the CUNY Master of Science, Data Analytics program. As part of my final project during the Fall 2014 semester, I acquired and analyzed <a href="http://wonder.cdc.gov/natality-current.html">Natality (birth) data from the U.S. Deparment of Health and Human Services (HHS)</a> in combination with <a href="http://data.bls.gov/timeseries/LNS14000000">Unemployment data from the U.S. Bureau of Labor Statistics</a> to discover possible relationships.</p> 2014-12-31T00:00:00+00:00 http://dwdii.github.io/2014/12/31/Fair-Market-Rent-Data-Analysis <h2 id="fair-market-rent-data-analyzer">Fair Market Rent Data Analyzer</h2> <p><a href="https://github.com/dwdii/HudFmrDataAnalyzer"><img style="position: absolute; top: 0; right: 0; border: 0;z-index:2000" src="https://camo.githubusercontent.com/a6677b08c955af8400f44c6298f40e7d19cc5b2d/68747470733a2f2f73332e616d617a6f6e6177732e636f6d2f6769746875622f726962626f6e732f666f726b6d655f72696768745f677261795f3664366436642e706e67" alt="Fork me on GitHub" data-canonical-src="https://s3.amazonaws.com/github/ribbons/forkme_right_gray_6d6d6d.png" /></a> As part of my work in the CUNY Master of Science, Data Analytics program, I developed a small application to load and analyze <a href="http://www.huduser.org/portal/datasets/fmr.html">Fair Market Rent (FMR) data from the U.S. Department of Housing and Urban Development (HUD)</a>. The application, simply named HudFmrDataAnalyzer, is written in Python, and is just chipping at the iceberg of how rents compare between localities in the United States, and also how they are changing in recent years. A light weight 1 year future forecast using linear regression is included as well.</p> 2014-11-17T00:00:00+00:00 http://dwdii.github.io/2014/11/17/Julia-Studio-UnitRange-has-no-field-len <h2 id="julia-studio-error-type-unitrange-has-no-field-len">Julia Studio ERROR type UnitRange has no field len</h2> <p>Are you getting flacky behavior from Julia Studio? I did. I’m on Windows 8.1 and every time I tried to run a jl script it either would do nothing, or I would get the error in the title of this post (and maybe the script would run). If I reset the console via the Tools &gt; Julia &gt; Reset Console menu item I could usually get another execution out of the debugger. The Julia Studio Console didn’t seem to work with REPL commands either.</p> 2014-10-24T00:00:00+00:00 http://dwdii.github.io/2014/10/24/DNScymbal-New-Version <h2 id="dnscymbal-updated-for-dnsimple-api-v1">DNScymbal Updated for DNSimple API v1</h2> <p>I received a tweet about a month ago from <a href="https://twitter.com/dnsimple">@DNSimple</a>:</p> 2014-09-27T00:00:00+00:00 http://dwdii.github.io/2014/09/27/Data-Set-Profile-Bike-Sharing-Demand <h2 id="data-set-profile-bike-sharing-demand">Data Set Profile Bike Sharing Demand</h2> <p>As you may know I recently started pursuing my <a href="http://bit.ly/cunymsda">Master of Science, Data Analytics</a> at the <a href="http://www.cuny.edu">City University of New York (CUNY)</a>. One of the courses I am taking currently uses the <a href="http://r-project.org">R language</a> among other technologies to acquire, clean and analyze various data set. During one of my projects I chose to analyze the bike sharing rental data set which is part of the <a href="https://www.kaggle.com/c/bike-sharing-demand">Bike Sharing Demand Kaggle competition</a>.</p> 2014-09-01T00:00:00+00:00 http://dwdii.github.io/2014/09/01/Book-Review-R-For-Everyone <h2 id="book-review---r-for-everyone">Book Review - R for Everyone</h2> <p>I recently started reading the book <a href="http://www.amazon.com/dp/0321888030">R for Everyone: Advanced Analytics and Graphics</a> by <a href="http://twitter.com/jaredlander">Jared Lander</a>. Although I haven’t finished reading the book yet, I’ve read enough to have developed an initial reaction that I thought I would share.</p> 2013-11-03T00:00:00+00:00 http://dwdii.github.io/2013/11/03/Data-Movement-w-Partition-Switching <h2 id="data-movement-with-partition-switching-in-sql-server">Data Movement with Partition Switching in SQL Server</h2> <p>This post is a follow-on to my prior “part 1” post <a href="/2013/09/12/SQL-Server-Table-Partitioning.html">Table Partitioning in SQL Server</a> where I introduced the topic and laid-out the data definition which will be the basis for this post on data movement with partition switching in SQL Server.</p> 2013-09-12T00:00:00+00:00 http://dwdii.github.io/2013/09/12/SQL-Server-Table-Partitioning <h2 id="table-partitioning-in-sql-server">Table Partitioning in SQL Server</h2> <p>Very large tables can be a bear to load, query and archive. How is “very large table” defined? Usually in hundreds of gigabytes or terabytes. Regardless, if you have a given table which is so large as to be causing maintenance or query bottlenecks, you might benefit from table partitioning.</p> 2013-08-28T00:00:00+00:00 http://dwdii.github.io/2013/08/28/GitHub-Pages-Jekyll-Ampersands <h2 id="github-pages-jekyll-ampersands-and-apostrophes">GitHub Pages, Jekyll, Ampersands and Apostrophes</h2> <p>Why am I getting this “page build failed” error on my GitHub Pages checkin? I have been asking myself that question lately. I’ve been migrating my blog into GitHub Pages, but haven’t taken the time to setup Ruby and Jekyll on my Windows 7 dev machine. <a href="https://help.github.com/articles/pages-don-t-build-unable-to-run-jekyll#syntax-errors">GitHub Pages Help recommends running the Jekyll build locally</a> to troubleshoot these errors, but the <a href="http://jekyllrb.com/docs/installation/">“official documentation does not support installation on Windows platforms”</a>.</p> 2013-08-20T00:00:00+00:00 http://dwdii.github.io/2013/08/20/First-Post <h2 id="my-first-jekyll-post">My First Jekyll Post!</h2> <p>What can I say? This is my first simple blog post using the Jekyll engine on Github Pages.</p> 2013-07-22T00:00:00+00:00 http://dwdii.github.io/2013/07/22/SammyJS-and-Mustache-in-ASP.NET <h2 id="sammyjs-and-mustache-in-aspnet---short-and-sweet">SammyJS and Mustache in ASP.NET - Short and Sweet</h2> <p>I recently started working with <a href="http://sammyjs.org/">SammyJS</a> in my investment tracking web app and thought others might get some value out of a blog post on the topic. I had already developed a basic app API and ASP.NET website, but wanted to add support for the Single Page App (SPA) model to facilitate mobile access.</p> 2012-03-28T00:00:00+00:00 http://dwdii.github.io/2012/03/28/MapRedux-PowerShell-BigData <h2 id="mapredux---powershell-and-big-data">MapRedux - PowerShell and Big Data</h2> <p>Have you been hearing about “big data”, “map reduce” and other large scale computing terms over the past couple of years and been curious to dig into more detail? Have you read some of the <a href="http://hadoop.apache.org/">Apache Hadoop</a> online documentation and unfortunately concluded that it wasn’t feasible to setup a “test” hadoop environment on your machine? More recently, I have read about some of Microsoft’s work to enable <a href="https://www.hadooponazure.com/">Hadoop on the Azure cloud</a>. Being a “Microsoft”-leaning technologist, I am more inclinded to be successful with experimentation when on the Windows platform. Of course, it is not that I am “religious” about one set of technologies other another, but rather more experienced.</p> 2011-10-23T00:00:00+00:00 http://dwdii.github.io/2011/10/23/Using-HTML5-Canvas-tag-for-Simple-Video-Animation <h2 id="using-html5-canvas-tag-for-simple-video-animation">Using HTML5 Canvas tag for Simple Video Animation</h2> <p>I recently picked up a wireless network camera to use around the house. After getting it up and running, I realized that the “mobile” support was less than ideal. The camera’s on-board web server provided a still image, an ActiveX control or a Java Applet as viewing options and only the still image was usable on my iPhone. I decided to try a webpage which would refresh automatically every 1 second in order to get a crude motion video effect via the HTML meta tag, as shown below.</p> 2011-05-27T00:00:00+00:00 http://dwdii.github.io/2011/05/27/Part-2-Automating-a-Visual-Studio-Build-with-PowerShell <h2 id="part-2---automating-a-visual-studio-build-with-powershell">Part 2 - Automating a Visual Studio Build with PowerShell</h2> <p>In <a href="/2011/05/20/Automating-a-Visual-Studio-Build-with-PowerShell-Part-1.html">my prior blog post on this topic</a>, I showed how to use the PowerShell <code class="language-plaintext highlighter-rouge">Start-Process</code> command to initiate the rebuild of a Visual Studio solution using MSBUILD. With today’s post, I’ll show how I’ve wrapped this call into a PowerShell function to make it more PowerShell friendly.</p> 2011-05-20T00:00:00+00:00 http://dwdii.github.io/2011/05/20/Automating-a-Visual-Studio-Build-with-PowerShell-Part-1 <h2 id="automating-a-visual-studio-build-with-powershell---part-1">Automating a Visual Studio Build with PowerShell - Part 1</h2> <p>One of the several hats I wear at work is one of configuration manager for the internal data management application my employer uses. This app currently has 11 solutions of active code which need to be built whenever we release to the QA environment. The QA releases typically occur a minimum of 3 times per month and take approximately 30 minutes from source control “Get Latest” to the final step of the deployment onto the QA servers.</p> 2011-05-06T00:00:00+00:00 http://dwdii.github.io/2011/05/06/How-to-Get-Exception-Info-in-PowerShell <h2 id="how-to-get-error-exception-details-in-powershell-catch-block">How to Get Error Exception Details in PowerShell Catch Block</h2> <p>For the last couple of weeks I have been required to import a data log file (CSV) from a legacy SCADA system into my centralized data management system. It occured to me today that I might be able to use PowerShell to automate some of the importing steps which include import to a DEV, QA and Production system. I’m still working on this script, but in the process of developing it, I had a need to learn more about an exception that was occuring in the script. I had my try catch block already specified, but was simply using <code class="language-plaintext highlighter-rouge">Write-Error</code>, as in the following code sample, to report a generic message “Failed to…” whenever the catch block was executed.</p>