BlueFire Data Solutions

I recently received my pre-ordered copy of Pentaho Solutions:  Business Intelligence and Data Warehousing with Pentaho and MySQL (linky (amazon.com)) and felt it needed to be the first official BlueFireDS Book Review.  It is the first resource available from a third-party about the excellent BI suite Pentaho (pentaho.com).  If you have not worked with Pentaho before and are interested in BI or wanting to find a way to set up a BI system give it a look see.  I have worked with the Pentaho suite since version 1.6 and the project has evolved leaps and bounds through to the most recent version 3.5.

By far my largest quips with Pentaho was the lack of good third-party documentation, which I feel Pentaho Solutions fills in nicely.  Not only does the book give a thorough overview of Pentaho (from installation/setup to how to use the various components of the suite), it also is a great introduction into the BI field, with an overview on building and maintaining data warehouses.

Though I consider this a must-buy for anyone who works with Pentaho or any of the various tools that make up the full BI suite (including some very nice data tools outside of the suite  like DataCleaner and Power*Architect), there are a few quips that I must point out.  First, the code that is provided on the book’s page over at Wiley (see below for link) does not match sometimes with the descriptions in the book, or does not run outright when trying to setup the provided source system and data warehouse databases.  Reading through the code, I discovered that the databases and the correct permissions just needed to be created/setup for the databases to be setup.  Also, with some of the transformations used with the Pentaho Data Integrator (formerly known as Kettle) either had extra steps or fields that caused a small bit of confusion.

To be honest, these quips are typical with technical books mainly because of the way publishing cycles work.  They are easy to overlook considering the vast area covered by the text as well as how much the reader is able to get out of it.  Pentaho Solutions is definitely one of those resources I’ll be keeping close at hand for months to come.

By the way Roland and Jos, if either of you happen to read this, I’m looking forward to seeing if this will turn into a nice series of references.  It definitely has potential!

For completeness’ sake, here’s the table of contents, from the publisher’s site:

Introduction.

Part I Getting Started with Pentaho.

Chapter 1 Quick Start: Pentaho Examples.

Chapter 2 Prerequisites.

Chapter 3 Server Installation and Configuration.

Chapter 4 The Pentaho BI Stack.

Part II Dimensional Modeling and Data Warehouse Design.

Chapter 5 Example Business Case: World Class Movies.

Chapter 6 Data Warehouse Primer.

Chapter 7 Modeling the Business Using Star Schemas.

Chapter 8 The Data Mart Design Process.

Part III ETL and Data Integration.

Chapter 9 Pentaho Data Integration Primer.

Chapter 10 Designing Pentaho Data Integration Solutions.

Chapter 11 Deploying Pentaho Data Integration Solutions.

Part IV Business Intelligence Applications.

Chapter 12 The Metadata Layer.

Chapter 13 Using The Pentaho Reporting Tools.

Chapter 14 Scheduling, Subscription, and Bursting.

Chapter 15 OLAP Solutions Using Pentaho Analysis Services.

Chapter 16 Data Mining with Weka.

Chapter 17 Building Dashboards.

Index.

Tags: Pentaho, Pentaho Solutions

So recently I posted about how symfony has made the move to officially support Doctrine as the default ORM in future releases of my favorite PHP framework.  A lot of questions were raised, mainly because even though Propel is still going to be supported by symfony, it looked like Propel was dead in the water with not much active development taking place.

Interestingly enough, when I happened to look on the Propel website, I noticed a Propel version 1.4 and that got me wondering.  After some hunting, it turns out a blast from symfony’s past is now heading the development of the project.  That’s right folks, Francois Zaninotto is now working on revamping Propel (see here (groups.google.com) for more details).

I’m glad to see that a good ORM is getting someone with a lot of good ideas leading the forefront.  I’m very interested in seeing what will be coming out of the gate with the revamped version 2.0.

Tags: propel, Symfony

I’m noticing that quite a few folks are posting about how MySQL is going to die a slow and painful death now that Sun has been purchased by Oracle.  As I commented on one site – I have to politely disagree.  The MySQL franchise may very well go away or be rebranded (which I think would be a dumb move on Oracle’s part, given the large community, and significant market share that MySQL has), but the code itself won’t be able to be locked away, at least in the most current form.

Already there are two main forks of the MySQL server:

• Drizzle – which is focused on cloud computing and web based applications
• MariaDB – which is the MySQL server code upgraded with the Maria storage engine, headed by Monty.

Now having forks can cause significant rifts within any Open Source project’s community, but on the bright side no matter what happens with Oracle, the MySQL community does have alternatives that are in some way compatible.  The same goes for many of the other projects that Sun has worked on and has been open-sourcing, most likely in preparation of being bought out.

I’m cautiously optimistic that Oracle won’t do anything too rash to the Sun/primary MySQL iteration and will work work to strengthening the ties to the MySQL community.

Just wanted to make a quick post about some recent work I’ve been doing with the end goal being my first plugin for symfony.  A customer recently requested an event calendar for their site, so I figured what better way to build one that with symfony?  Currently, the calendar has a month view.  I’m working now on integrating the events into the calendar.  Once that has been accomplished, I’ll release the first beta.

My ultimate goal for the plugin is to have a daily, weekly, monthly, and yearly views with events viewed based off of user security level (which I am using the sfGuardPlugin for user security).  I have been working off and on for the last week or so to get the calendar into shape.

Once I get the event system integrated, I’ll post some pics.

In other news, I’ve been working on various tutorials that I really need to finish up.  I’ll try to have one out on symfony AJAX integrations soon.

Tags: sfAgendaCalPlugin, Symfony

So at work I have been developing my symfony skills by rebuilding some of the database systems I’ve worked on in the past (first up:  update the Training Database….).  I know that with symfony 1.2, javascript will be uncoupled from the framework to allow for each developer to work with their favorite javascript framework (jQuery here =p) instead of being forced into Prototype/script.aculo.us helpers.

I’m excited to see what javascript development this will bring in with seperate plugins for each of the major javascript frameworks – with this making the symfony project even stronger than before.

What does this all mean for BlueFire?  Well, for starters I’m changing direction with the example software for the site and working with symfony as the primary framework for my projects.  Look out for some samples up in the coming weeks =D

Tags: javascript, Symfony

Seeing that there’s been interest with Magento, I figured I would write up my experiences since the last time I published about it.

First off, Magento does a fairly good job about documenting the basics about the program.  As I found out last time, it will take a bit of digging to get into super-specific problems – mine just happened to be a host issue that had to be worked around.

So basically, to access the administration side of the system, you need to go to the root directory of where you placed Magento and add /admin to the end of the address.  In my case, I actually created a store folder to keep it separate from the rest of the website so the address would look similar to:  http://somesite.com/store/admin.

Now during the installation process, you created an administrator account.  Using that will get you access to the backend – which happens to be a nice sales dashboard (one of the features that really sold me on Magento is the reporting capabilities build in, not sure if other OSS e-commerce solutions do something similar.  Feel free to post a comment and let me know what’s up with the other solutions.  Heck, if you have a good review of any of them, send it to me too =p).

One of my big quips about Magento at this point is that even with my broadband connection, Magento seems to take it’s time loading initially, which I sincerely hope they work on in future releases.  Also at first brush, it seems a bit complex to build addon modules for Magento, which I have also seen comments about elsewhere when reading reviews about php|architect’s Magento guide.

I do like how the menu is organized, with the ability to create promotions for a particular date range, newsletter creation, and the ability to create personalized themes for stores (haven’t worked on creating one yet, but I do like that they give the option).

Anywho, I’ll keep playing around with Magento (when I’m not delving into symfony =p) and see what else I can come up with.

Tags: Magento

Just wanted to make a quick post saying congrats to the Mozilla team for successfully hitting the world record for the most software downloads in a 24 hour period, as well as making the most kick-ass browsing experience I’ve ever had possible!

I’ve been using Firefox 3 since before Download Day (btw – got your certificate for downloading FF3 yet?  If not, clicky) and I must applaud the Mozilla group for all the hard work they put into the new version, and must say the plugin community really put the petal to the metal to ensure that most of the popular plugins worked with our new browser friend.

Anywho, I’ve kinda been out of the picture for a while due to my back/other things taking precident, but I’m still working on a few things coming up.  I hope I’ll be able to finish at least one of them up this week for posting, we’ll see.

Tags: Firefox Mozilla

I pledged – will you? =p

Tags: Download Day, Firefox, Mozilla

So I’m starting to work with e-Commerce systems, and picked Magento because it has a very robust feature set (I still want to check out Zen Cart, for it also looks promising) . I’ve about got my current customer’s store up, just waiting on their host to setup the MySQL database to finish the install. Since I’ve got a bit of time to kill before that’s complete, I figured I’d do a small writeup on how much fun GoDaddy brought to me lol.

Okay, so all you really need to download is the Magento version that you want to use. Fortunately, GoDaddy finally caught up with the times and now provides PHP 5.2 with it’s hosting packages. Otherwise, you get to have fun with the “CGI last resort fix“.

So I also found out that because GoDaddy hosts both PHP 4 AND PHP 5, developers have to change the file extensions from .php to .php5 (what fun, right?).

So, hunting around the Internet brought me a quick solution for this.

In your Magento folder/setup, there should already be a .htaccess file. Open that file up and add the following line:

AddHandler x-httpd-php5 .php

Now when you go to the Magento index page, you should see a ‘No Input File Specified’ error. This is another easy fix.

If you do not already have a php.ini file in your main folder, you need to create one. This will only work if you have selected the option of using Hosting Connection 2.0 . This will allow you to modify server settings for your site.

I had to name my file ‘php5.ini’ for it to recognize. I then added the following line:

cgi.fix_pathinfo = 1

GoDaddy decided to behave at this point and allowed me to see the installation wizard. Do not despair if it doesn’t work – there’s a few more things that can be done, just check out the official posting about this issue here.

So I only had one more error to pop up after this. It came in the form of a nice long text box. Apparently when I FTP’d the Magento software, one file didn’t transfer over. If this happens, just place the file where it belongs based off of the error message you may receive.

So once that’s all done, you should also have Magento hosted on a GoDaddy account.

As I explore the features of Magento, I’ll post my thoughts and ideas/tips up here.

Tags: GoDaddy, Magento

So a large chunk of my daily job consists of finding better ways of viewing and obtaining data. Being the geeky individual I am, I do my best to push Open Source initiatives (as a real quick aside, anyone else had the extreme fun of having to prove over and over again that Open Source can be high quality, enterprise-grade stuff? I’m pretty sure that is the case in a lot of places, but it’s good to hear the horror stories lol).

So over the last two years I’ve come across many programs (Pentaho and Open Flash Chart to name a couple…) and have recently come across a really neat little php app over at sparkline.org: an open source application to build sparklines from databases/array data.

What are these sparklines anyway?

I was first introduced to the concept of sparklines by reading Information Dashboard Design: The Effective Visual Communication of Data by Stephen Few (perceptualedge.com). If you’re looking for a good overview on how to design information dashboards, I highly recommend picking it up.

Sparklines were developed by Edward Tufte, who describes them as “data-intense, design-simple, word-sized graphics” (here’s a post Tufte made about them).

Basically, they are a way to show a large chunk of data in a relatively small space. Here’s a few examples from sparkline.org:

Notice there are two types – a line chart and a bar chart. Both types can be embedded within a section of text to give quick, relevant data without breaking the stride of the writing. As the first example also shows, you can also have sparklines be any size depending on what information you want to display. I’ve been using them for On Time In Full calculations by customer – basically an overview page showing all customers on file with three sparklines beside them: one for On Time, In Full, and On Time In Full respectfully. This way everyone (and I do mean everyone – I’m all about keeping data open to all those involved with the process) within the facility can see how well we are at shipping to our customers on time.

I’ve gotten a tutorial about halfway finished on how to use the sparkline package with a mysql database (quick hint – since OTIF is a hit/miss calculation, use a MySQL case statement to transform your data into a hit/miss total….). Look for it soon.

EDIT – Now here’s some irony.  While looking through the WordPress widget repository, I came across this:  SparkStats Widget (widgets.wordpress.com).  It even uses the Sparkline PHP Graphing Library.  Check it out.

Tags: Edward Tuft, On Time In Full (OTIF), Open Source, sparklines, SparkStats Widget, Stephen Few