Ctools Icon Set

Our UX team, tired of putting up to the speed where we develop new ctools, decided that it would be a good idea to build an official icon set that you can get from our website.

Obviously, we allow it to be available to anyone, so feel free to (ab)use it!

Experimenting and sharing CCC2 charts

CCC, now on version 2, is a very powerful charting engine, with a lot of customization abilities. Like Thomas Morgner likes to quote, with great power comes great responsibility.  While a lot is possible, also a lot is not easy to do, and requires some knowledge.

The CCC website

The team made a huge effort to document CCC2. The website and the CCC jsdocs, complemented by Protovis Jsdocs, provides a great amount of resources for knowing how to best solve a problem

JsFiddle

There's also a great, well known, resource for sharing experiments with javascript, JsFiddle. We can also setup a CCC2 playground that people can use and fork to play with.

Here's a fiddle with a sample of CCC2 barchart with annotations.

A new challenge - Webdetails joins Pentaho

The Announcement

Now here's a blog post that a while back I wouldn't even think about writing.

I'm very happy to announce that Webdetails will join Pentaho! Here's an excerpt from the press release:

ORLANDO, FLA - April 22, 2013 – Delivering the future of analytics, Pentaho announced today that it has completed the acquisition of its Portugal-based consulting partner Webdetails. Pentaho will benefit from Webdetails’ visual interface development expertise and international consulting services provided by its 20-strong team. Webdetails’ founder Pedro Alves is a high-profile member of Pentaho’s open source community and will take on the new role of Senior VP, Community for Pentaho.

All the team at webdetails feel incredibly proud for the vote of confidence, and very excited to keep working as hard we can but now with the new goal of improving the pentaho platform and the community ecosystem. Oh! - wait, we've been doing that for the past 5 years! :p

The daily work of the now Portuguese pentaho crew won't change much - we'll still operate and provide services under the the name Webdetails (now with an additional tag a pentaho company), have our UX team rock and rolling and keep creating the ctools until we run out of letters on the alphabet.

We'll just have the huge opportunity of doing this things at a much larger scale. The plan to conquer the world advances as planned!

 

The Community

I had the pleasure to make the announcement first hand in the Pentaho Day 2013, the amazing Brazilian community event that had over 180 people that chose to spend a Saturday on a room with a faulty A/C talking about a subject that unites them.

I'll take the new role of SVP of Community for Pentaho, where I'll try to be "the chief advocate and interface to Pentaho’s active open source developer community". As you may have guessed the bit on italics was copy-pasted directly from the aforementioned press release. I don't even know what that means!

As I had the opportunity to present to the audience in Brazil, the guys at pentaho that thought it would be a good idea to put me on that role have no idea what's going to happen!

This is not me

Building a community is not about going to events on tropical countries, drinking huge amounts of liquor and saying how cool pentaho is (wait, scratch the part about liquor and going to tropical countries, I do like that)

This is me!

A community doesn't exist because a company wants it to. A community is a reflex of the strategy of a company. People will only gather around a common cause - any cause - if they feel connected to it.

I'll be doing a lot of evangelization work, but not externally. It's an inside job, where I'll try to share what the community is doing, feeling, saying and fight to make that information reach the product and the strategy.

I do believe there's a lot to be done, and it's not an easy task to try to achieve a good balance between the Enterprise and Community objectives, but I do believe there's lots of room for improvement!

Please feel free to come to me for any suggestions, complaints, rants, whatever. Pentaho needs to make a good job in allowing people from the community to succeed. It's the only way this works. We're not a bunch of good guys. At least I'm not! Only when people feel they're getting something they'll be willing to give something back.

Community is all about creating Opportunity!


-pedro

New Ctools Releases: 13.03.25

Easter release!

CDF

CCC V2 integration!

Full changelog:

    * CCC V2 integration

    * Added capability to detect session timeout and request new credentials (instead of silently failing)

    * Feature: Support for extra options in select component    

    * Feature:  Add clickAction & expand selectors to table component

    * Fixed Heatgrid Sample

    * Fix: Multiselect component now checks for null values regardless of the plugin used.

    * Fix [REDMINE-1822] - Tooltip not showing in select components

    * Fix [REDMINE-309] - td.expandingClass now has a dummy click for it to work as intended.

    * Fix: Column format in table component now takes hidden columns into account. Ex: If column 0 is hidden: - Now: format for column 1 is colFormats[1] - Previously: format for column 1 was colFormats[0]

CDE

Support for CCC V2

Full changelog:

    * Support for CCC V2

    * Update siteMap component to accept an url from where to fetch the siteMap json using an ajax call.

    * Implemented [Redmine-93] -  Close the Popup Component

    * Fixed [Redmine-1840] - External resource editing is failing

    * Fixed [Redmine-345] - Scrollbars in PopupComponent

    * Fixed [Redmine-1874] -  Remplate import should now work as intended 

CDA

Full changelog:

    * Fixed [REDMINE-1851] - Added conditions that protect the TableComponent from empty metadata queries (eg. Select include mode with output options [1,0] and an empty query)

    * Fixed [REDMINE-1881] - IntegerArrays not working on sql queries

    * Reintroduced deprecated SQLReportDataFactory.setQuery for pentaho 3.9 support

GGG

    * Support for CCC v2

    * Internal architectural changes - refactored cgg to cgg-core and cgg-pentaho.

CDV

Full changelog:

* There were some edge cases where parameters were not being correctly parsed in cdv's validation dashboard

Pentaho Day 2013 in Brazil, Fortaleza

I'm pretty excited about this. The Brazilian pentaho community, one of the most active ones I know of, is organizing the second community event in Brazil, this time in Fortaleza, April 20

It's gonna be as great as all the other ones. Registrations can be done here and discussion space will be in the pentahobrasil facebook page.

The event agenda will be shared later, as we're giving the community the opportunity to talk about their own experiences.

This is a joint organization of all pentaho consulting companies in Brazil in an amazing display of cooperation. I'll be there! :)

CPK - Community Plugin Kickstarter - Request For Comments

CPK

One more Ctool in the works. Once again I'm calling out for comments, suggestions, whatever. Let me know what you thing and suggestions for improvements, or even if you think this makes no sense at all.

Motivation

Pentaho is very well known for being a very good Business Analytics software, but is in fact much more than that; Pentaho is a great platform to build on top of.
Using an easy analogy, I see Pentaho acting as an operating sytem where people can build Application on top of

Objective

The goal of CPK is to provide a simple and easy way to develop pentaho plugins that behave like packaged applications, simplifying it's structure.
The UI is built using CDE, with a simple methodology to create new dashboards / pages and a sitemap that provides simple navigation and a default template
There are 3 options for doing server-side code:

1. Kettle transformations
2. Javascript server side code execution
3. Java classes

The first two are recommended, since it's easier to register new endpoints just by dropping code in a directory and no compilation is necessary.
With this approach not only we expect to make it easier and faster to develop plugins, we also hope to lower down the specific technical requirements to build them. The end goal is that business consultants are able to build new plugins, not requiring specific java knowledge.

Structure

This is the resulting plugin structure. Ideally, no compilation is necessary, so everything except maybe the lib directory could be stored in a VCS system.
This is the proposed stub configuration
to be completed

CPK_Plugin
|-- conf
|-- dashboards
|-- endpoints
|-- lib
`-- plugin.xml

CPK administrative features

Besides providing the regular templating for creating new plugins, CPK can have an administrative UI with the following features:

• List existing plugins
• Detect if the plugins are up to date with the latest version of CPK
• Allow the creation of new plugins
• Allow to change plugin metadata
• List and register new endpoints (UI and code)

Here's a list of stretch goals / nice to have

• Import UI from existing dashboards in solution
• Allow editing dashboards from this UI
• Submit marketplace metadata to Pentaho
• Generate distribution zip package

Updates

As the CPK framework or any of it's dependencies gets improved, the plugins themselves can't stay outdated. There will be a version information attached to the CPK plugin version so that it's possible to upgrade to the latest version.

Dependencies

CPK will have as little code as possible, making it as simple as possible to develop plugins. However, it will need a few dependencies:

• Pentaho
• CPF - Community Plugin Framework, with the common set of code for the plugins
• CDE - Community Dashboard Editor
• CDF - Community Dashboard Framework
• CDA - Community Data Access

Link with Pentaho Marketplace

Once a plugin is developed, and the authors think it's in a state that can be shared, CPK will be able to generate a packaged plugin and metadata information so it can be integrated into Pentaho's marketplace.
Pentaho will then be able to categorize / approve the plugin so that it becomes available to other users through the marketplace

License

This project uses MPLv2

CBF - Build RPMs automatically

Motivation

CBF - Community Build Framework - is an amazing tool for being able to maintain several projects, compile pentaho and even doing remote deploys using rsync.

However, there are cases where rsync is not the best way to maintain a server installation. At Mozilla, IT uses Puppet to maintain everything related to automated server maintenance. The way CBF is structured allows us to define rules to automate solution checkouts, but we need to automate the installation of 3 extra bits:

• Pentaho webapp
• Pentaho solution (the system/ and admin/ directories)
• Administration / Enterprise console

Approach

The cluster uses RHEL6 distribution, so we chose to pack the distributions using rpm format. This should work for any rpm-based distribution. If someone wants to do the same for deb or something else, just get in touch or simply fork CBF on github and send a pull request.

Building RPMs through CBF

CBF now has 2 extra options (be sure to grab the latest version from github):

 create-rpm-dist - Creates a tgz file with pentaho, pentaho-style and ROOT folders - to be used with a puppet script
 create-rpms - Creates rpms based on the specs found on the config/rpm folder for the solution

 In order for this to be usable, we need to create a directory called rpm under config, and place there the .spec files for the rpm's we're creating. We will also need a specific makefile.

This is the structure:

project-metrics
├── config
│   ├── build-ee-pedro.properties
│   ├── build-pedro.properties
│   ├── build.properties
│   └── rpm
│       ├── makefile
│       ├── webdetails-pentaho-adminconsole.spec
│       ├── webdetails-pentaho-adminconsole-stage.spec
│       ├── webdetails-pentaho-solution.spec
│       ├── webdetails-pentaho-solution-stage.spec
│       ├── webdetails-pentaho-webapp.spec
│       └── webdetails-pentaho-webapp-stage.spec
├── etl
├── patches
│   ├── project-metrics
│   ├── target-build
│   └── target-dist
├── patches-ee
│   ├── enterprise-console
│   ├── server
│   └── solution
└── solution -> ../project-metrics-solution

 You'll see that I'm building here 6 RPMs, for a production and stage environment.

You can download the entire set of files from here.

Here's a sample of the file webdetails-pentaho-solution.spec:

Name:  Webdetails-Pentaho-Webapp
Version: @VERSION@
Release: 1%{?dist}
Summary: Pentaho webapp customized for Mozilla metrics
Group:  Applications/Engineering
License: MPL-1.0
Source:     webdetails-pentaho.tgz


%description

%prep
rm -rf %{buildroot}
%build
tar zxvf ../SOURCES/webdetails-pentaho.tgz 

%install
mkdir %{buildroot}
make DESTDIR=%{buildroot} install

%clean
rm -rf %{buildroot}

%files
%defattr(-,root,root,-)
%doc

/opt/pentaho/pentaho-server/server/webapps/pentaho
/opt/pentaho/pentaho-server/server/webapps/pentaho-style
/opt/pentaho/pentaho-server/server/webapps/ROOT

%changelog

And here's the makefile that is called by the specs. I chose to install the pentaho server under /opt/pentaho/, but this can be changed to any desired path:

all: install
    
install:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server/server/
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps
 cp -r pentaho $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/pentaho
 cp -r pentaho-style $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/pentaho-style    
 cp -r ROOT $(DESTDIR)/opt/pentaho/pentaho-server/server/webapps/ROOT    

install-solution:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-solution
 cp *.* $(DESTDIR)/opt/pentaho/pentaho-solution/
 cp -r system $(DESTDIR)/opt/pentaho/pentaho-solution/system
 cp -r admin $(DESTDIR)/opt/pentaho/pentaho-solution/admin


install-stage:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage/server  
 mkdir $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps
 cp -r pentaho $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/pentaho
 cp -r pentaho-style $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/pentaho-style    
 cp -r ROOT $(DESTDIR)/opt/pentaho/pentaho-server-stage/server/webapps/ROOT    

install-solution-stage:
 mkdir $(DESTDIR)/opt
 mkdir $(DESTDIR)/opt/pentaho
 mkdir $(DESTDIR)/opt/pentaho/pentaho-solution-stage
 cp *.* $(DESTDIR)/opt/pentaho/pentaho-solution-stage/ 
 cp -r system $(DESTDIR)/opt/pentaho/pentaho-solution-stage/system
 cp -r admin $(DESTDIR)/opt/pentaho/pentaho-solution-stage/admin

Dependencies 

Note that this is depends on having rpmbuild installed on your system. Would work on linux and mac, and even eventually on windows with cygwin.

After running the create-rpms ant target, you should get the RPMs in the dist/rpm/RPMS/ directory.

Next steps

You probably know what you want to do with the RPMs. You can either install them directly, or if you're using puppet you may want to set up a yum repository. Lots of tutorials on how to do this, here's one.

Packaging Enterprise Edition

By default, this targets know how to handle the CE version. We can change a few options and tell it to pack the Enterprise Edition, just by setting some extra options pointing to a different location where we unpacked the EE version:

######################################
## EE BASED RPM BUILD
######################################

rpm.source.webapp = /home/pedro/tex/pentaho/ee/4.8/biserver-ee/tomcat/webapps/
rpm.source.solution = /home/pedro/tex/pentaho/ee/4.8/biserver-ee/pentaho-solutions/
rpm.source.administration-console = /home/pedro/tex/pentaho/ee/4.8/enterprise-console