tag:blogger.com,1999:blog-66443296935303004672024-03-28T05:47:05.565+00:00Pedro Alves on Business IntelligenceEyes on the Pentaho planet, feet on the groundPedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.comBlogger279125tag:blogger.com,1999:blog-6644329693530300467.post-50828853375640805462021-08-02T17:00:00.001+01:002021-08-02T17:00:00.211+01:00Pentaho 9.2 is available<h1 style="text-align: left;"> Pentaho 9.2 is available</h1><div style="text-align: left;"><span face=""open sans", sans-serif" style="background-color: white; caret-color: rgb(119, 119, 119); color: #777777;">Short and sweet - I'll make it fast.</span></div><div style="text-align: left;"><br /></div><h2 style="text-align: left;"><span lang="EN-US">Main features</span></h2><div><ul style="text-align: left;"><li>Expand to Microsoft Azure – SQL DB, ADLS, HDInsight</li><li>Updated Data Fabric – CDP and MapR (HPE Ezmeral Data Fabric)</li><li>Productivity Boost – Logging and Upgrade</li></ul></div><div><span lang="EN-US"><div><span face=""open sans", sans-serif" style="background-color: white; caret-color: rgb(119, 119, 119); color: #777777;"><br /></span></div><div><span face=""open sans", sans-serif" style="background-color: white; caret-color: rgb(119, 119, 119); color: #777777;">Go get EE through the </span><a href="https://support.pentaho.com/hc/en-us" style="color: #954f72; display: inline; font-family: "Open Sans"; outline: none;">support portal</a><span face=""open sans", sans-serif" style="background-color: white; caret-color: rgb(119, 119, 119); color: #777777;">, and CE in the </span><a href="https://sourceforge.net/projects/pentaho/" style="color: #954f72; display: inline; font-family: "Open Sans"; outline: none;">usual place</a><span face=""open sans", sans-serif" style="background-color: white; caret-color: rgb(119, 119, 119); color: #777777;">!</span></div><div><span face=""open sans", sans-serif" style="background-color: white; caret-color: rgb(119, 119, 119); color: #777777;"><br /></span></div></span></div><div style="text-align: left;"><br /></div>Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com10tag:blogger.com,1999:blog-6644329693530300467.post-63951946379406337222020-10-06T10:52:00.001+01:002020-10-06T10:52:05.424+01:00Pentaho 9.1 is available!<p></p><div class="separator" style="clear: both; text-align: left;"><style class="WebKit-mso-list-quirks-style">
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:0cm;
margin-left:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"open sans",sans-serif;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
h1
{mso-style-priority:9;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-link:"Heading 1 Char";
mso-style-next:Normal;
margin-top:12.0pt;
margin-right:0cm;
margin-bottom:0cm;
margin-left:0cm;
mso-pagination:widow-orphan lines-together;
page-break-after:avoid;
mso-outline-level:1;
font-size:16.0pt;
font-family:"Calibri Light",sans-serif;
mso-ascii-font-family:"Calibri Light";
mso-ascii-theme-font:major-latin;
mso-fareast-font-family:"Yu Gothic Light";
mso-fareast-theme-font:major-fareast;
mso-hansi-font-family:"Calibri Light";
mso-hansi-theme-font:major-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:major-bidi;
color:#2F5496;
mso-themecolor:accent1;
mso-themeshade:191;
mso-font-kerning:0pt;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;
font-weight:normal;}
p.MsoTitle, li.MsoTitle, div.MsoTitle
{mso-style-priority:10;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-link:"Title Char";
mso-style-next:Normal;
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:0cm;
margin-left:0cm;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:28.0pt;
font-family:"Calibri Light",sans-serif;
mso-ascii-font-family:"Calibri Light";
mso-ascii-theme-font:major-latin;
mso-fareast-font-family:"Yu Gothic Light";
mso-fareast-theme-font:major-fareast;
mso-hansi-font-family:"Calibri Light";
mso-hansi-theme-font:major-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:major-bidi;
letter-spacing:-.5pt;
mso-font-kerning:14.0pt;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
p.MsoTitleCxSpFirst, li.MsoTitleCxSpFirst, div.MsoTitleCxSpFirst
{mso-style-priority:10;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-link:"Title Char";
mso-style-next:Normal;
mso-style-type:export-only;
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:0cm;
margin-left:0cm;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:28.0pt;
font-family:"Calibri Light",sans-serif;
mso-ascii-font-family:"Calibri Light";
mso-ascii-theme-font:major-latin;
mso-fareast-font-family:"Yu Gothic Light";
mso-fareast-theme-font:major-fareast;
mso-hansi-font-family:"Calibri Light";
mso-hansi-theme-font:major-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:major-bidi;
letter-spacing:-.5pt;
mso-font-kerning:14.0pt;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
p.MsoTitleCxSpMiddle, li.MsoTitleCxSpMiddle, div.MsoTitleCxSpMiddle
{mso-style-priority:10;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-link:"Title Char";
mso-style-next:Normal;
mso-style-type:export-only;
margin:0cm;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:28.0pt;
font-family:"Calibri Light",sans-serif;
mso-ascii-font-family:"Calibri Light";
mso-ascii-theme-font:major-latin;
mso-fareast-font-family:"Yu Gothic Light";
mso-fareast-theme-font:major-fareast;
mso-hansi-font-family:"Calibri Light";
mso-hansi-theme-font:major-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:major-bidi;
letter-spacing:-.5pt;
mso-font-kerning:14.0pt;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
p.MsoTitleCxSpLast, li.MsoTitleCxSpLast, div.MsoTitleCxSpLast
{mso-style-priority:10;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-link:"Title Char";
mso-style-next:Normal;
mso-style-type:export-only;
margin:0cm;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:28.0pt;
font-family:"Calibri Light",sans-serif;
mso-ascii-font-family:"Calibri Light";
mso-ascii-theme-font:major-latin;
mso-fareast-font-family:"Yu Gothic Light";
mso-fareast-theme-font:major-fareast;
mso-hansi-font-family:"Calibri Light";
mso-hansi-theme-font:major-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:major-bidi;
letter-spacing:-.5pt;
mso-font-kerning:14.0pt;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
mso-themecolor:hyperlink;
text-decoration:underline;
text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}
p
{mso-style-noshow:yes;
mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"open sans",sans-serif;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"open sans",sans-serif;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"open sans",sans-serif;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
mso-add-space:auto;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"open sans",sans-serif;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
span.Heading1Char
{mso-style-name:"Heading 1 Char";
mso-style-priority:9;
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:"Heading 1";
mso-ansi-font-size:16.0pt;
mso-bidi-font-size:16.0pt;
font-family:"Calibri Light",sans-serif;
mso-ascii-font-family:"Calibri Light";
mso-ascii-theme-font:major-latin;
mso-fareast-font-family:"Yu Gothic Light";
mso-fareast-theme-font:major-fareast;
mso-hansi-font-family:"Calibri Light";
mso-hansi-theme-font:major-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:major-bidi;
color:#2F5496;
mso-themecolor:accent1;
mso-themeshade:191;}
span.TitleChar
{mso-style-name:"Title Char";
mso-style-priority:10;
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:Title;
mso-ansi-font-size:28.0pt;
mso-bidi-font-size:28.0pt;
font-family:"Calibri Light",sans-serif;
mso-ascii-font-family:"Calibri Light";
mso-ascii-theme-font:major-latin;
mso-fareast-font-family:"Yu Gothic Light";
mso-fareast-theme-font:major-fareast;
mso-hansi-font-family:"Calibri Light";
mso-hansi-theme-font:major-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:major-bidi;
letter-spacing:-.5pt;
mso-font-kerning:14.0pt;}
p.dt, li.dt, div.dt
{mso-style-name:dt;
mso-style-unhide:no;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
p.dd, li.dd, div.dd
{mso-style-name:dd;
mso-style-unhide:no;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Times New Roman",serif;
mso-fareast-font-family:"Times New Roman";}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:#1000;
mso-fareast-language:EN-US;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;
mso-header-margin:35.4pt;
mso-footer-margin:35.4pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:15468018;
mso-list-type:hybrid;
mso-list-template-ids:-1589983932 1445203756 1902564074 -778009950 302669314 -912516644 1285612514 -405660418 -955855778 -384250164;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l0:level2
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:‒;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Times New Roman",serif;}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:•;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Arial",sans-serif;
mso-bidi-font-family:"Times New Roman";}
@list l1
{mso-list-id:408313899;
mso-list-type:hybrid;
mso-list-template-ids:898022220 134807553 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2
{mso-list-id:894901128;
mso-list-type:hybrid;
mso-list-template-ids:-422786482 -1214635814 -1018530044 836815896 1481657070 29631906 541646562 936173986 2118648738 -1745563636;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level2
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:‒;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Times New Roman",serif;}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l3
{mso-list-id:1337615407;
mso-list-template-ids:1290706028;}
@list l3:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level2
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l3:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l4
{mso-list-id:2127387917;
mso-list-type:hybrid;
mso-list-template-ids:-141940538 -1721729876 1187795952 -1473583986 -1282777654 2043186876 1598697330 -1767058488 1256639538 231605956;}
@list l4:level1
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l4:level2
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:‒;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Times New Roman",serif;}
@list l4:level3
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l4:level4
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l4:level5
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l4:level6
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l4:level7
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l4:level8
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l4:level9
{mso-level-number-format:bullet;
mso-level-text:§;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
-->
</style><p class="MsoTitle"><span lang="EN-US">Pentaho 9.1 is available<o:p></o:p></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US">It’s that time of the year! A new release is available!<o:p></o:p></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"><br /></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US">Go get EE through the <a href="https://support.pentaho.com/hc/en-us">support portal</a>, and CE in the <a href="https://sourceforge.net/projects/pentaho/files/Pentaho%209.1/">usual place</a>!</span></p><h1><span lang="EN-US">Main features<o:p></o:p></span></h1><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google Data Proc Support<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Catalog Steps in Spoon<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">New Upgrade Utility<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-ansi-language: EN-US; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">And a bunch of consolidation stuff:<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="margin-left: 72.0pt; mso-add-space: auto; mso-list: l1 level2 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: "Courier New"; mso-ansi-language: EN-US; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">20+ Continuous Improvements<o:p></o:p></span></p><p class="MsoListParagraphCxSpMiddle" style="margin-left: 72.0pt; mso-add-space: auto; mso-list: l1 level2 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: "Courier New"; mso-ansi-language: EN-US; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">10+ Platform Updates<o:p></o:p></span></p><p class="MsoListParagraphCxSpLast" style="margin-left: 72.0pt; mso-add-space: auto; mso-list: l1 level2 lfo1; text-indent: -18.0pt;"><!--[if !supportLists]--><span lang="EN-US" style="font-family: "Courier New"; mso-ansi-language: EN-US; mso-fareast-font-family: "Courier New";">o<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">200+ Performance/quality bugs<o:p></o:p></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><h1><span lang="EN-US">Google Data Proc<o:p></o:p></span></h1><p style="background-color: white; font-size: medium; line-height: 20.3pt; margin-bottom: 0cm; margin-top: 15pt;"><span style="color: #333333; font-family: "open sans", sans-serif;">You can now access and process data from a Google Dataproc cluster in PDI. Google Dataproc is a cloud-native Spark and Hadoop managed service that has built-in integration with other Google Cloud Platform services, such as BigQuery and Cloud Storage. With PDI and Google Dataproc, you can migrate from on-premise to the Google Cloud.<o:p></o:p></span></p><p style="background-color: white; font-size: medium; font-variant-ligatures: normal; line-height: 20.3pt; margin-bottom: 0cm; margin-top: 15pt; orphans: 2; widows: 2;"><span style="color: #333333; font-family: "open sans", sans-serif;">You can use PDI's Google Dataproc driver and named connection feature to access data on your Google Dataproc cluster as you would other Hadoop clusters, like Cloudera and Amazon EMR. See <a data-scope="local" href="https://help.pentaho.com/Documentation/9.1/Setup/Set_up_the_Pentaho_Server_to_connect_to_a_Hadoop_cluster"><span style="color: #cc0000;">Set up the Pentaho Server to connect to a Hadoop cluster</span></a> for further instructions.<o:p></o:p></span></p><p class="MsoNormal" style="font-size: medium;"><o:p> </o:p></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l2 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><b><span lang="EN-US">What’s New</span></b><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">New Hadoop driver</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">AEL-Spark support</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l2 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><b><span lang="EN-US">Version:</span></b><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Google Dataproc - 1.4 (Ubuntu 18.04 LTS, Hadoop 2.9, Spark 2.4)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l2 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><b><span lang="EN-US">Benefit</span></b><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Enables processing large data sets in Google Data Proc clusters</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">On-premise data movement/migration</span><o:p></o:p></p><p class="MsoNormal" style="font-size: medium; margin-left: 36pt;"><o:p> </o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l2 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Hadoop Driver supports the following:</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Multi-cluster</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">HDFS</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Hive</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">PMR Hive</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Oozie</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Sqoop</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Hadoop Job Executor</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Pig</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l2 level2 lfo2; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Parquet / Avro / ORC </span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l2 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">VFS support for GCS</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l2 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Hbase is not supported </span><o:p></o:p></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><h1><span lang="PT">Lumada Data Catalog steps for PDI<o:p></o:p></span></h1><p style="background-color: white; font-size: medium; line-height: 20.3pt; margin-bottom: 0cm; margin-top: 15pt;"><span style="color: #333333; font-family: "open sans", sans-serif;"><a href="https://help.pentaho.com/Documentation/9.1/Products/PDI_and_Lumada_Data_Catalog"><span style="color: #cc0000;">Lumada Data Catalog</span></a> lets data engineers, data scientists, and business users accelerate metadata discovery and data categorization, and permits data stewards to manage sensitive data. Data Catalog collects metadata for various types of data assets and points to the asset's location in storage. Data assets registered in Data Catalog are known as data resources.<o:p></o:p></span></p><p style="background-color: white; font-size: medium; font-variant-ligatures: normal; line-height: 20.3pt; margin-bottom: 0cm; margin-top: 15pt; orphans: 2; widows: 2;"><span style="color: #333333; font-family: "open sans", sans-serif;">You can use the folllowing four new PDI steps to work with Data Catalog metadata and data resources within your PDI transformations:<o:p></o:p></span></p><p class="dt" style="background-color: white; background-position: initial initial; background-repeat: initial initial; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm; text-indent: -18pt;"><!--[if !supportLists]--><span style="color: #333333; font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-bidi-font-weight: bold; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><b><span style="color: #333333; font-family: "open sans", sans-serif;"><a data-scope="local" href="https://help.pentaho.com/Documentation/9.1/Products/Read_Metadata"><span style="color: #cc0000;">Read Metadata</span></a><o:p></o:p></span></b></p><p class="dd" style="background-color: white; font-size: medium; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm;"><span style="color: #333333; font-family: "open sans", sans-serif;">Search Data Catalog’s existing metadata for specific data resources, including their storage location.<o:p></o:p></span></p><p class="dt" style="background-color: white; background-position: initial initial; background-repeat: initial initial; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm; text-indent: -18pt;"><!--[if !supportLists]--><span style="color: #333333; font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-bidi-font-weight: bold; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><b><span style="color: #333333; font-family: "open sans", sans-serif;"><a data-scope="local" href="https://help.pentaho.com/Documentation/9.1/Products/Write_Metadata"><span style="color: #cc0000;">Write Metadata</span></a><o:p></o:p></span></b></p><p class="dd" style="background-color: white; font-size: medium; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm;"><span style="color: #333333; font-family: "open sans", sans-serif;">Revise the existing Data Catalog tags associated with an existing data resource.<o:p></o:p></span></p><p class="dt" style="background-color: white; background-position: initial initial; background-repeat: initial initial; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm; text-indent: -18pt;"><!--[if !supportLists]--><span style="color: #333333; font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-bidi-font-weight: bold; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><b><span style="color: #333333; font-family: "open sans", sans-serif;"><a data-scope="local" href="https://help.pentaho.com/Documentation/9.1/Products/Catalog_Input"><span style="color: #cc0000;">Catalog Input</span></a><o:p></o:p></span></b></p><p class="dd" style="background-color: white; font-size: medium; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm;"><span style="color: #333333; font-family: "open sans", sans-serif;">Reads the CSV text file types or Parquet data formats of a Data Catalog data resource that is stored in a Hadoop or S3 ecosystem and outputs the data payload in the form of rows to use in a transformation.<o:p></o:p></span></p><p class="dt" style="background-color: white; background-position: initial initial; background-repeat: initial initial; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm; text-indent: -18pt;"><!--[if !supportLists]--><span style="color: #333333; font-family: Symbol; font-size: 10.0pt; mso-bidi-font-family: Symbol; mso-bidi-font-size: 12.0pt; mso-bidi-font-weight: bold; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><b><span style="color: #333333; font-family: "open sans", sans-serif;"><a data-scope="local" href="https://help.pentaho.com/Documentation/9.1/Products/Catalog_Output"><span style="color: #cc0000;">Catalog Output</span></a><o:p></o:p></span></b></p><p class="dd" style="background-color: white; font-size: medium; line-height: 20.3pt; margin-bottom: 0cm; margin-left: 36pt; margin-top: 0cm;"><span style="color: #333333; font-family: "open sans", sans-serif;">Encodes CSV text file types or Parquet data formats using the schema defined in PDI to create a new data resource or to replace or update an existing data resource in Data Catalog.<o:p></o:p></span></p><p class="MsoNormal" style="font-size: medium;"><o:p> </o:p></p><p class="MsoNormal" style="font-size: medium;"><o:p> </o:p></p><h1><span lang="EN-US">New Upgrade utility<o:p></o:p></span></h1><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l0 level1 lfo3; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Arial",sans-serif; mso-fareast-font-family: Arial;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Current Scope: </span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l0 level2 lfo3; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Scope only 9.0 to 9.1 (coming later: will extend to 8.3 LTS)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l0 level1 lfo3; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Arial",sans-serif; mso-fareast-font-family: Arial;">•<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Reliable upgrades and rollback</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l0 level2 lfo3; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Initial environment check to detect product component and will only upgrade what is there</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l0 level2 lfo3; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">White list to persist customization</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l0 level2 lfo3; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Will persist all plug-ins across upgrade</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l0 level2 lfo3; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Automatically whitelist all database driver jars</span><o:p></o:p></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><h1>Compatibility Updates<o:p></o:p></h1><p class="MsoNormal" style="font-size: medium;"><o:p> </o:p></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US">This:<o:p></o:p></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><div class="separator" style="clear: both; text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgz7_C1IL4dWpd-3uw6sxLF62jiUX1xZtoc6jN6qAnCiWC8BD4q93uDUQFxJDZfGEFVuJasik-2SaQd1h3q9iBw76MXLnnKzamr9fi9hH8AtT7Sc4x0_SnA2zo1oEOkUEDHgqJ7NNSWsb0/s840/Picture+1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="424" data-original-width="840" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgz7_C1IL4dWpd-3uw6sxLF62jiUX1xZtoc6jN6qAnCiWC8BD4q93uDUQFxJDZfGEFVuJasik-2SaQd1h3q9iBw76MXLnnKzamr9fi9hH8AtT7Sc4x0_SnA2zo1oEOkUEDHgqJ7NNSWsb0/s320/Picture+1.png" width="320" /></a></div><br /><p class="MsoNormal" style="font-size: medium;"><br /></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><h1><span lang="EN-US">Other improvements:<o:p></o:p></span></h1><p class="MsoNormal" style="font-size: medium;"><o:p> </o:p></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US">This:<o:p></o:p></span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l4 level1 lfo4; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Data Integration</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">S3 Multipart Upload now allow configurable part sizes (</span><a href="https://jira.pentaho.com/browse/PDI-16606"><span lang="EN-US">PDI-16606</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">MongoDB Plug-in now allows PLAIN credentials for LDAP integration (</span><a href="https://jira.pentaho.com/browse/PDI-17228"><span lang="EN-US">PDI-17228</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l4 level1 lfo4; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Dashboards / Reporting</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">10-100x performance improvement for certain large slices and roll-ups for Mondrian Cubes (</span><a href="https://jira.pentaho.com/issues/?jql=key%20in%20(MONDRIAN-2683%2C%20MONDRIAN-2687%2C%20MONDRIAN-2688%2C%20MONDRIAN-2689%20)"><span lang="EN-US">JIRA Link</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Option to remove/hide the filter panel when used in a dashboard (</span><a href="https://jira.pentaho.com/browse/ANALYZER-2270"><span lang="EN-US">ANALYZER-2270</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Count and Count Distinct Summary on currency fields uses the default format (</span><a href="https://jira.pentaho.com/browse/PIR-699"><span lang="EN-US">PIR-699</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Admins can now customize the template(s) used for exporting to PDF and Excel (</span><a href="https://jira.pentaho.com/browse/ANALYZER-12"><span lang="EN-US">ANALYZER-12</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 36.0pt; mso-list: l4 level1 lfo4; tab-stops: list 36.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: Wingdings; mso-bidi-font-family: Wingdings; mso-fareast-font-family: Wingdings;">§<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Platform</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Passwords stored in the BA Server config files and repository are now encrypted (</span><a href="https://jira.pentaho.com/browse/BISERVER-3497"><span lang="EN-US">BISERVER-3497</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="margin-left: 72.0pt; mso-list: l4 level2 lfo4; tab-stops: list 72.0pt; text-indent: -18.0pt;"><!--[if !supportLists]--><span style="font-family: "Times New Roman",serif; mso-fareast-font-family: "Times New Roman";">‒<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; line-height: normal;"> </span></span><!--[endif]--><span lang="EN-US">Users are now able to change their own password (</span><a href="https://jira.pentaho.com/browse/BISERVER-13699"><span lang="EN-US">BISERVER-13699</span></a><span lang="EN-US">)</span><o:p></o:p></p><p class="MsoNormal" style="font-size: medium;"><o:p> </o:p></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US">-pedro </span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p><p class="MsoNormal" style="font-size: medium;"><span lang="EN-US"> </span></p></div><br /><br /><p></p>Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com48tag:blogger.com,1999:blog-6644329693530300467.post-41366012244276274972020-02-04T14:30:00.003+00:002020-02-05T17:25:31.811+00:00Pentaho 9.0 is available<h2>
<div class="MsoTitle">
<span style="font-size: 22pt; letter-spacing: 0.25pt; text-indent: -18pt;">Pentaho 9.0 is available</span><br />
<span style="font-size: 22pt; letter-spacing: 0.25pt; text-indent: -18pt;"><br /></span><span style="font-size: small; font-weight: normal; letter-spacing: 0.25pt; text-indent: -18pt;">Without further ado: <a href="https://www.hitachivantara.com/en-us/products/data-management-analytics/pentaho-platform.html">Get Enterprise Edition here</a>, and get <a href="https://community.hitachivantara.com/s/article/downloads">Community Edition here</a></span></div>
</h2>
<h1 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc31040144"></a><a href="https://www.blogger.com/null" name="_Toc22562219"><span style="mso-bookmark: _Toc31040144;">PDI Multi Cluster
Hadoop Integration</span></a><o:p></o:p></h1>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc22562220"></a><a href="https://www.blogger.com/null" name="_Toc11922540"><span style="mso-bookmark: _Toc22562220;">Capability</span></a><o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">Pentaho connects to
Cloudera Distribution for Hadoop (CDH), Hortonworks Data Platform (HDP), Amazon
Elastic MapReduce (EMR). Pentaho also supports many related services such as
HDFS, HBase, Oozie, Zookeeper, and Spark. <o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">Before this
release, Pentaho Server as well as PDI design time environment – Spoon, can
work with only one Hadoop cluster at a time. It required multiple
transformations, instances, and pipelines to execute multiple Hadoop clusters.
With 9.0 release, major architecture changes have occurred to easily configure,
connect and manage multiple Hadoop clusters.<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Users can access and process data from multiple
Hadoop clusters from different distros and versions- all from single
transformation and instance of Pentaho. <o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Also, within Spoon, users can now set up three
distinct cluster configs, all having reference to the
specific cluster, without having to restart Spoon. There is
also a new configuration UI to easily configure your Hadoop drivers for
managing different clusters.<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Improved cluster configuration experience and
secure connection with the new UI<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Supports following distros: Hortonworks HDP
v3.0, 3.1; Cloudera CDH v6.1, 6.2; Amazon EMR v5.21, 5.24.<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<span style="font-weight: normal;"><span style="font-size: small;"><br /></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<span style="font-weight: normal;"><span style="font-size: small;">Existing single
cluster/shim functionality will continue to work. <span style="mso-spacerun: yes;"> </span><o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<span style="font-weight: normal;"><span style="font-size: small;"><br /></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<span style="font-weight: normal;"><span style="font-size: small;">The following example
shows the Multi-cluster implemented in the same data pipeline via connecting to
both Hortonworks HDP and Cloudera CDH clusters.</span></span><o:p></o:p></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimAj9vkgtdryNDmzEMMGU6nfnizAnBI2gdeaq5gnmS_hS2eK_Tz-bj8D6BBQbnZbZB4SectDN_6smlO3FVskuV_b8NtWEY_7hFBLEVnpI2Tcu9OQoFM3Excdc1grGLRqpODrIC2BdZ9To/s1600/Picture+1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="289" data-original-width="779" height="147" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEimAj9vkgtdryNDmzEMMGU6nfnizAnBI2gdeaq5gnmS_hS2eK_Tz-bj8D6BBQbnZbZB4SectDN_6smlO3FVskuV_b8NtWEY_7hFBLEVnpI2Tcu9OQoFM3Excdc1grGLRqpODrIC2BdZ9To/s400/Picture+1.png" width="400" /></a></div>
<div class="MsoListBulletCxSpLast" style="tab-stops: 36.0pt;">
<br />
<o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc11922541"><span style="mso-bookmark: _Toc22562221;"><br /></span></a></h3>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc22562221"></a><a href="https://www.blogger.com/null" name="_Toc11922541"><span style="mso-bookmark: _Toc22562221;">Use Cases and
Benefits</span></a><o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Enables hybrid big data processing support
(on-prem or cloud)- all within single pipeline <o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpLast" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Simplifies Pentaho’s integration with Hadoop
clusters including enhanced UX of cluster configurations</span></span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc22562222"></a><a href="https://www.blogger.com/null" name="_Toc11922542"><span style="mso-bookmark: _Toc22562222;">Key Considerations</span></a><o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-weight: normal;"><span style="font-size: small;">Adaptive Execution Layer Spark isn’t validated
to execute pipelines connecting to multiple Hadoop clusters.<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpLast" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Pentaho Map Reduce isn’t validated to execute
pipelines connecting to multiple Hadoop clusters. </span></span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc22562223"></a><a href="https://www.blogger.com/null" name="_Toc11922543"><span style="mso-bookmark: _Toc22562223;">Additional
Resources</span></a><o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">See <a href="https://help.pentaho.com/Documentation/9.0/Work_with_data/Adding_a_new_driver">Adding
a new driver</a> for how to add a driver. See <a href="https://help.pentaho.com/Documentation/9.0/Work_with_data/Connecting_to_a_Hadoop_cluster_with_the_PDI_client">Connecting
to a Hadoop cluster with the PDI client</a> for how to create a named
connection.<a href="https://www.blogger.com/null" name="_Toc22562235"><o:p></o:p></a></span></div>
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<span style="font-size: small; font-weight: normal;"><span style="mso-bookmark: _Toc22562235;"><!--[if !supportLists]--><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;"> </span></span></span><!--[endif]-->Follow
the suggestions in the </span><a href="https://help.pentaho.com/Documentation/9.0/Setup/Big_Data_issues"><span style="mso-bookmark: _Toc22562235;">Big Data issues</span></a><span style="mso-bookmark: _Toc22562235;"> troubleshooting sections to help resolve
common issues when working with Big Data and Hadoop, especially </span><a href="https://help.pentaho.com/Documentation/9.0/Setup/Big_Data_issues#Legacy_mode_activated_when_named_cluster.0D.0A____configuration_cannot_be_located"><span style="mso-bookmark: _Toc22562235;">Legacy mode activated when named cluster
configuration cannot be located</span></a></span><span style="mso-bookmark: _Toc22562235;"><span style="font-size: small; font-weight: normal;">.</span><o:p></o:p></span></div>
</div>
</h2>
<h1 style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="mso-bookmark: _Toc22562235;"><a href="https://www.blogger.com/null" name="_Toc31040145">PDI AEL-Spark Enhancements</a></span><o:p></o:p></h1>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Capability<o:p></o:p></h3>
<h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnBxlBtiV8ywyPrTr13r50lM9WLNST53U8P2IzHOzliu00Jtnpo7j7sXzlJy_DLu1q7Ie1_5xhnV0EjWgtPC1qmpPO4TIzXrj32OVciwlu_1Sk2h0HR9BXiY3ZUdKmbF3EFcrThrKXevo/s1600/Picture+2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="335" data-original-width="727" height="183" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgnBxlBtiV8ywyPrTr13r50lM9WLNST53U8P2IzHOzliu00Jtnpo7j7sXzlJy_DLu1q7Ie1_5xhnV0EjWgtPC1qmpPO4TIzXrj32OVciwlu_1Sk2h0HR9BXiY3ZUdKmbF3EFcrThrKXevo/s400/Picture+2.png" width="400" /></a></div>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The Pentaho
Adaptive Execution Layer (AEL) is intended to provide flexible and transparent
data processing with Spark, in addition to the native Kettle engine. The goal
of AEL is to develop complex pipelines visually and then execute in Kettle or
Spark based on data volume and SLA requirements.<span style="mso-spacerun: yes;"> </span>AEL allows PDI users to designate Spark as
execution engine for their transformations apart from Kettle.<o:p></o:p></span></div>
<span style="font-size: small; font-weight: normal;"><br /></span>
<br />
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;"><br /></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The v9.0.0 release
includes the following performance and flexibility enhancements to AEL-Spark:<o:p></o:p></span></div>
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Step level spark specific performance tuning
options<o:p></o:p></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Enhanced logging configuration and information
entered into PDI logs<o:p></o:p></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Added support for Spark 2.4, with existing 2.3
support <o:p></o:p></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Supports following distros: Hortonworks HDP
v3.0, 3.1; Cloudera CDH v6.1, 6.2; Amazon EMR v5.21, 5.24.<o:p></o:p></span></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<span style="font-size: small; font-weight: normal;"><br /></span></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<span style="font-size: small; font-weight: normal;">The following example
showcases Spark App and Spark Tuning on specific steps within a PDI
transformation: </span><o:p></o:p></div>
<div class="MsoListBulletCxSpMiddle" style="tab-stops: 36.0pt;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYrQJRKnjfeDVDF_04tpa6y7GJr46vr-z1osLh7pD9TyczHxfQXv56n-mEhZrYtF6AKSblaR1MAuzqGpQK5bctmpSNdgqpFl5BhlXO2KV8WF3kIYJiG9Xs7fV_gJhs5F94dbfQpDLt0m4/s1600/Picture+3.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="366" data-original-width="739" height="197" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhYrQJRKnjfeDVDF_04tpa6y7GJr46vr-z1osLh7pD9TyczHxfQXv56n-mEhZrYtF6AKSblaR1MAuzqGpQK5bctmpSNdgqpFl5BhlXO2KV8WF3kIYJiG9Xs7fV_gJhs5F94dbfQpDLt0m4/s400/Picture+3.png" width="400" /></a></div>
<div class="MsoListBulletCxSpLast" style="margin-left: 0cm; mso-add-space: auto; tab-stops: 36.0pt; text-indent: 0cm;">
<span style="mso-spacerun: yes;"> </span><span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shape id="Picture_x0020_3"
o:spid="_x0000_i1029" type="#_x0000_t75" style='width:355pt;height:176pt;
visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:////Users/pedalves/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_image003.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Use Cases and Benefits<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Eliminates black box feel with better visibility<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Enable advanced Spark users with tools to improve
performance</span></span><o:p></o:p></div>
<div class="MsoListBulletCxSpLast" style="margin-left: 0cm; mso-add-space: auto; tab-stops: 36.0pt; text-indent: 0cm;">
<br /></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Key Considerations<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">Users must be aware
of the following additional items related to AEL v9.0.0:<o:p></o:p></span></div>
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Spark v2.2 is not supported.<o:p></o:p></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Native HBase steps are only available for CDH
and HDP distributions.<o:p></o:p></span></div>
<div class="MsoListBulletCxSpLast" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Spark 2.4 is the highest Spark version currently
supported. </span></span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Additional Resources<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">See the following
documentation for more details: <a href="https://help.pentaho.com/Documentation/9.0/Setup/About_Spark_tuning_in_PDI">About
Spark Tuning in PDI</a>, <a href="https://help.pentaho.com/Documentation/9.0/Setup/Spark_Tuning">Setup Spark
Tuning</a>, <a href="https://help.pentaho.com/Documentation/9.0/Setup/Configuring_application_tuning_parameters_for_Spark">Configuring
Application Tuning Parameters for Spark</a></span></span><span class="MsoHyperlink"> <o:p></o:p></span></div>
</div>
</h2>
<h1 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc31040146">Virtual
File System (VFS) Enhancements</a><o:p></o:p></h1>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Capability<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<!--[if !supportLists]--><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-weight: normal;"><span style="font-size: small;">The changes to the VFS are in two main areas:<o:p></o:p></span></span></div>
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->1. We added Amazon S3 and Snowflake Staging as VFS
providers to named VFS Connections and introduced the Pentaho VFS (pvfs) that
can reference defined VFS Connections and their protocols. In the S3 protocol,
we support S3A and Session Tokens in 9.0.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->The general format of a Pentaho VFS URL is:</span></span><br />
<span style="font-weight: normal;"><span style="font-size: small;">pvfs://VFS_Connection/path (including a namespace, bucket or similar)<o:p></o:p></span></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXNNQenmHmMm_jy_0DXoXt63y35DF-79XPyzRef-I-EtWF4mV0d5Was6cIeuNbAijaYo1DAzegu3jXLN6BCaB-SX4W6oY24T99Eb39rrossoHGcXs_u79twk6cx8ZwAsU0BL1RngTpTwU/s1600/Picture+4.png" imageanchor="1" style="font-weight: normal; margin-left: 1em; margin-right: 1em;"><span style="font-size: small;"><img border="0" data-original-height="354" data-original-width="1004" height="140" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjXNNQenmHmMm_jy_0DXoXt63y35DF-79XPyzRef-I-EtWF4mV0d5Was6cIeuNbAijaYo1DAzegu3jXLN6BCaB-SX4W6oY24T99Eb39rrossoHGcXs_u79twk6cx8ZwAsU0BL1RngTpTwU/s400/Picture+4.png" width="400" /></span></a></div>
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<span style="font-weight: normal;"><span style="font-size: small;"><br /></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;"><span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shape id="Picture_x0020_7"
o:spid="_x0000_i1028" type="#_x0000_t75" style='width:483pt;height:170pt;
visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:////Users/pedalves/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_image004.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></span></span></div>
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->2. A new file browsing experience has been
added. The enhanced VFS browser allows users to browse any preconfigured VFS
locations using named connections, their local filesystem, configured clusters
via HDFS, as well as a Pentaho repository, if connected. <span style="mso-spacerun: yes;"> </span></span></span><o:p></o:p></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDleUxtZ7q3HgF00Z-SUJtDnZo55ctiSg16oBlYTcnmMZn7StvUWhXN_9VYrhNslov_iS5W-atW6qKGDn4pQnMzncBl9_rQNqX-S6kBHs8X7MyUFIis3fzGn6nmPY11K3wpd_3U6YxgL0/s1600/Picture+5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="647" data-original-width="972" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDleUxtZ7q3HgF00Z-SUJtDnZo55ctiSg16oBlYTcnmMZn7StvUWhXN_9VYrhNslov_iS5W-atW6qKGDn4pQnMzncBl9_rQNqX-S6kBHs8X7MyUFIis3fzGn6nmPY11K3wpd_3U6YxgL0/s400/Picture+5.png" width="400" /></a></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<br />
<span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shape id="Picture_x0020_10"
o:spid="_x0000_i1027" type="#_x0000_t75" style='width:468pt;height:312pt;
visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:////Users/pedalves/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_image005.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Use Cases and Benefits<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<!--[if !supportLists]--><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span style="font-size: small; font-weight: normal;">Through the support of Pentaho VFS, you have an
abstraction of the protocol. That means, when you want to change your provider
in the future, all your jobs and transformations work seamless after this
change in the VFS Connection. Today, you reference S3. Tomorrow, you want to
reference another provider, for example HCP or Google Cloud. Using Pentaho VFS,
your maintenance burden in these cases is much lower.<o:p></o:p></span></div>
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->VFS Connections also enables you to use different
accounts and servers (including namespaces, buckets or similar) within one PDI
transformation. Example: You want to process data within one transformation
from S3 with different buckets and accounts.<o:p></o:p></span></div>
<div class="MsoNormal" style="mso-list: l0 level1 lfo1;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Combining named VFS connections with the new
file browsing experience provides a convenient way to easily access remote
locations and extend the reach of PDI. The new file browser also offers the
ability to manage files across those remote locations. For example, a user can
easily copy files from Google Cloud into an S3 bucket using the browser's copy
and paste capabilities. A user can then easily reference those files using
their named connections, in supported steps and job entries. <o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;"><span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shape id="Picture_x0020_8"
o:spid="_x0000_i1026" type="#_x0000_t75" style='width:480pt;height:319pt;
visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:////Users/pedalves/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_image006.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBZnRLy57MyUNL5W8De0LTfc6kRF8jkopgrxshh7dlS2ejH_IE4YKOCcsNHz1ZmthdExvbyETAxlYjd0tmB0ABxrmFPUBRV1ZlIZ8nQcKSObGsatFWeBxv8U8UUmPZ_PYPaiCn5aYuxxM/s1600/Picture+6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><span style="font-size: small; font-weight: normal;"><img border="0" data-original-height="662" data-original-width="999" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhBZnRLy57MyUNL5W8De0LTfc6kRF8jkopgrxshh7dlS2ejH_IE4YKOCcsNHz1ZmthdExvbyETAxlYjd0tmB0ABxrmFPUBRV1ZlIZ8nQcKSObGsatFWeBxv8U8UUmPZ_PYPaiCn5aYuxxM/s400/Picture+6.png" width="400" /></span></a></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;"><br /></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">A user can manage
all files, whether they are local or remote in a central location. For example,
there is no need to login to the Amazon S3 Management Console to create
folders, rename, delete, move or copy files. Even a copy between the local
filesystem and S3 is possible and you can upload/download files from within
Spoon. <o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The new file
browser also offers capabilities such as search, which allows a user to find
filenames which match a specified search string. The file browser also remembers
a user's most recently accessed jobs and transformations for easy reference.</span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Key Considerations<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">As of PDI 9.0, the
following protocols are supported: Amazon S3, Snowflake Staging (read only),
HCP, Google CS<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The following steps
and job entries have been updated to use the new file open save dialog for 9.0:
Avro input, Avro output, Bulk load into MSSQL, Bulk load into MySQL, Bulk load
from MySQL, CSV File Input, De-serialize from file, Fixed File Input, Get data
from XML, Get file names, Get files rows count, Get subfolder names, Google
Analytics, GZip CSV input, Job (job entry), JSON Input, JSON Output, ORC input,
ORC output, Parquet Input, Parquet output, Text file output, Transformation
(job entry)<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The File / Open
dialog is still using the old browsing dialog. The new VFS browser for opening
jobs and transformations can be reached through the File / Open URL menu entry.<o:p></o:p></span></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">
Additional Resources<o:p></o:p></span></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">See <a href="https://help.pentaho.com/Documentation/9.0/Products/Virtual_File_System_connections">Virtual
File System connections</a>, <a href="https://commons.apache.org/proper/commons-vfs/filesystems.html">Apache
Supported File Systems</a> and <a href="https://help.pentaho.com/Documentation/9.0/Products/Work_with_transformations#t_pentaho_open_a_transformation_work_with_transformations_pdi">Open
a transformation</a> for more information.<o:p></o:p></span></div>
</div>
</h2>
<h1 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc31040147"><span style="font-size: large; font-weight: normal;">Cobol
copybook steps</span></a><span style="font-size: small; font-weight: normal;"><o:p></o:p></span></h1>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">
Capability<o:p></o:p></span></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">PDI now has two
transformation steps that can be used to read mainframe records from a file and
transform them into PDI rows.<o:p></o:p></span></div>
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]--><a href="https://help.pentaho.com/Documentation/9.0/Products/Copybook_Input#GUID-1A13E373-2BFD-4D14-963C-A810BE5D2A77">Copybook
input</a>: This step reads the mainframe binary data files that were originally
created using the copybook definition file and outputs the converted data to the
PDI stream for use in transformations.<o:p></o:p></span></div>
<div class="MsoListBulletCxSpLast" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]--><a href="https://help.pentaho.com/Documentation/9.0/Products/Read_metadata_from_Copybook#GUID-6DF7C90E-B8E6-4E70-9E49-EC9CACB1808F">Read
metadata from Copybook</a>: This step reads the metadata of a copybook definition
file to use with ETL Metadata Injection in PDI.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The Copybook steps
also support metadata injection, extended error handling and can work with
redefines. Extensive examples for these use cases are available in the PDI
samples folder.<o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtDhbcGDw_U-Cv948hjEtWUaiIcEcrGIEmCNG65fWrKZXr5311xaue0ONdgKvcOScGXNGTFAzEHg3WMn80RjetLM6wGBTlU5BMDJwxuY1_m44JSJCkc_cIY8JSsaCnp_8CHITdXk-D5Lg/s1600/Picture+7.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="220" data-original-width="904" height="96" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtDhbcGDw_U-Cv948hjEtWUaiIcEcrGIEmCNG65fWrKZXr5311xaue0ONdgKvcOScGXNGTFAzEHg3WMn80RjetLM6wGBTlU5BMDJwxuY1_m44JSJCkc_cIY8JSsaCnp_8CHITdXk-D5Lg/s400/Picture+7.png" width="400" /></a></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<br /></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shape id="Picture_x0020_11"
o:spid="_x0000_i1025" type="#_x0000_t75" style='width:435pt;height:107pt;
visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:////Users/pedalves/Library/Group%20Containers/UBF8T346G9.Office/TemporaryItems/msohtmlclip/clip_image007.png"
o:title=""/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Use Cases and Benefits<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">Pentaho Data
Integration supports simplified integration with fixed-length records in
mainframe binary data files, so that more users can ingest, integrate, and
blend mainframe data as part of their data integration pipelines. This capability
is critical if your business relies on massive amounts of customer and
transactional datasets generated in mainframes that you want to search and
query to create reports.</span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Key Considerations<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">This step works
with Fixed Length COBOL records only. Variable record types such as VB, VBS,
OCCURS DEPENDING ON are not supported.</span><o:p></o:p></div>
</div>
</h2>
<h3 style="tab-stops: 36.0pt; text-indent: 0cm;">
Additional Resources<o:p></o:p></h3>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">For more
information about using copybook steps in PDI, see <a href="https://help.pentaho.com/Documentation/9.0/Products/Copybook_steps_in_PDI#GUID-3132E04C-6AFC-4164-A76A-E9A5C9D03CFD">Copybook
steps in PDI</a></span><span class="MsoHyperlink"><o:p></o:p></span></div>
</div>
</h2>
<h1 style="tab-stops: 36.0pt; text-indent: 0cm;">
<a href="https://www.blogger.com/null" name="_Toc31040148"></a><a href="https://www.blogger.com/null" name="_Toc29381589"></a><a href="https://www.blogger.com/null" name="_Toc27770842"><span style="mso-bookmark: _Toc29381589;"><span style="mso-bookmark: _Toc31040148;">Additional Enhancements</span></span></a><o:p></o:p></h1>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
New Pentaho Server Upgrade
Installer<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">The Pentaho Server
Upgrade Installer is an easy to use graphical user interface that automatically
applies the new release version to your archive installation of the Pentaho
Server. You can upgrade versions 7.1 and later of the Pentaho Server directly
to version 9.0 using this simplified upgrade process via the user interface of
the Pentaho Server Upgrade Installer. <o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">See <a href="https://help.pentaho.com/Documentation/9.0/Setup/Upgrade_the_Pentaho_Server">Upgradethe Pentaho Server</a> for instructions.</span></span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Snowflake Bulk Loader improvement<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">The Snowflake Bulk
Loader has added support for doing a table preview in PDI 9.0. When connected
to Snowflake and on the Output tab, select a table in the drop-down menu. The preview
window is populated, showing the columns and data types associated with that
table. The user can see the expected column layout and data types to match up
with the data file.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">For more
information, please see the job entry documentation of the <a href="https://help.pentaho.com/Documentation/9.0/Products/Bulk_load_into_Snowflake">Snowflake
Bulk Loader</a>. </span></span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Redshift IAM security support and
Bulk load improvements<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">With this release,
you have more Redshift Database Connection Authentication Choices, these are<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Standard credentials (default) – user password<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->IAM credentials<o:p></o:p></span></span></div>
<div class="MsoListBulletCxSpLast" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-weight: normal;"><span style="font-size: small;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Profile located on local drive in AWS
credentials file <o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">Bulk load into Amazon
Redshift enhancements: New Options tab and Columns option in the Output tab of
the Bulk load into Amazon Redshift PDI entry. Use the settings on the Options
tab to indicate if all the existing data in the database table should be
removed before bulk loading. Use the Columns option to preview the column names
and associated data types within your selected database table.<o:p></o:p></span></span></div>
<div class="MsoListBullet" style="tab-stops: 36.0pt;">
<span style="font-weight: normal;"><span style="font-size: small;">See <a href="https://help.pentaho.com/Documentation/9.0/Products/Bulk_load_into_Amazon_Redshift">Bulk
load into Amazon Redshift</a> for more information.<o:p></o:p></span></span></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Improvements in AMQP and UX
changes in Kinesis<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The AMQP Consumer
step provides Binary message support, for example allowing to process AVRO
formatted data. <o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">Within the Kinesis
Consumer step, users can change the output field names and types.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">See the
documentation of the <a href="https://help.pentaho.com/Documentation/9.0/Products/AMQP_Consumer">AMQP
Consumer</a> and <a href="https://help.pentaho.com/Documentation/9.0/Products/Kinesis_Consumer">Kinesis
Consumer</a> steps for more details.</span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Metadata Injection (MDI)
Improvements<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">In PDI 9.0.0, we
continue to enable more steps to support metadata injection (MDI):<o:p></o:p></span></div>
<div class="MsoListBulletCxSpFirst" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Split Field to Rows<o:p></o:p></span></div>
<div class="MsoListBulletCxSpMiddle" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->Delete<o:p></o:p></span></div>
<div class="MsoListBulletCxSpLast" style="mso-list: l2 level1 lfo3;">
<!--[if !supportLists]--><span style="font-size: small; font-weight: normal;"><span style="font-family: "symbol"; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;"><span style="mso-list: Ignore;">·<span style="font-family: "times new roman"; font-stretch: normal; font-style: normal; font-variant: normal; line-height: normal;">
</span></span></span><!--[endif]-->String operations<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">In the Excel Writer
step, the missing MDI step option “Start writing at cell”, has been added. This
option can also be injected now.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">Additionally, the
metadata injection example is now available in the samples folder:</span><br />
<span style="font-size: small; font-weight: normal;">/samples/transformations/metadata-injection-example<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">See <a href="https://help.pentaho.com/Documentation/9.0/Products/ETL_metadata_injection">ETL
metadata injection</a> for more details.</span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Excel Writer: Performance
improvement<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The performance of
the Excel Writer has been drastically improved when using templates. A sample
test file with 40,000 rows needed about 90 seconds before 9.0 and now processes
in about 5 seconds.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">For further
details, please see PDI-18422.</span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
JMS Consumer changes<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">In PDI 9.0, we
added the following fields to the JMS Consumer step: MessageID, JMS timestamp
and JMS Redelivered.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">This addition
enables restartability and allows to omit duplicate messages.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">For further
details, please see PDI-18104 and the <a href="https://help.pentaho.com/Documentation/9.0/Products/JMS_Consumer">step
documentation</a>.<o:p></o:p></span></span></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Text file output: Header support
with AEL<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">You can set up the
Text file input step to run on the Spark engine via AEL. The Header option of
the Text file output step works now with AEL.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">For further
details, please see PDI-18083 and the <a href="https://help.pentaho.com/Documentation/9.0/Products/Using_the_Text_File_Output_step_on_the_Spark_engine">Using
the Text File Output step on the Spark engine</a> documentation.</span></span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Transformation & Job Executor
steps, Transformation & Job entries: UX improvement<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">Before 9.0, when passing
parameters to transformations/jobs, the options "Stream column name"
vs. "Value" ("Field to use" vs. "Static input
value") were ambiguous and led to hard to find issues. <o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">In 9.0, we added
behavior which prevents a user from entering values into both fields to avoid
these situations.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">For further
details, please see PDI-17974.</span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Spoon.sh Exit code improvement<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">Spoon.sh (that gets
called by kitchen.sh or pan.sh) sends the wrong exit status in certain
situations.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">In 9.0, we added a
new environment variable FILTER_GTK_WARNINGS to control this behavior for
warnings that effect the exit code. If the variable is set to anything, then a
filter is applied to ignore any GTK warnings. If you don’t want to filter any warnings,
then unset FILTER_GTK_WARNINGS.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">For further
details, please see PDI-17271.</span></span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Dashboard: Option for exporting
analyzer report into CSV format.<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">Now it's possible
to export an analyzer report into a CSF format file even when embedded on a
dashboard.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">In the previous
release the export option was available, but without the CSV format. <o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">The CSV format was
available only when using Analyzer outside dashboards, in this way we provide
functional parity between Analyzer standalone charts and charts embedded in
dashboards.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">For further
details, please see PDB-1327.</span></span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Analyzer: Use of date picker when
selecting ranges for a Fiscal Date level relative filter.<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">Before 9.0 and for
an AnalyzerFiscalDateFormat annotation on a level in a Time dimension, Analyzer
did not show the "Select from date picker" link.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">Now, relative dates
can be looked up from the current date on the Date level, then the date picker
can also be used to select the nearest fiscal time period.<o:p></o:p></span></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-weight: normal;"><span style="font-size: small;">For further
details, please see ANALYZER-3149.</span></span><o:p></o:p></div>
</div>
</h2>
<h2 style="tab-stops: 36.0pt; text-indent: 0cm;">
Mondrian: Option for setting the 'cellBatchSize'
default value.<o:p></o:p></h2>
<h2>
<div class="MsoTitle">
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">From a default
installation the mondrian.properties does not include
mondrian.rolap.cellBatchSize as a configurable property.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The purpose of this
improvement is to include this property in the mondrian.properties by default
in new builds so customers do not run into performance issues due to the
default value for this property being set too low. The default value of the
property should be clearly indicated in the properties file as well.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">The default value
has been updated to mondrian.rolap.cellBatchSize=1000000.<o:p></o:p></span></div>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
<span style="font-size: small; font-weight: normal;">This value was
chosen because this setting can run a very large 25M cell space report while
keeping total server memory usage around 6.7 GB which is under the 8GB we list
as the minimum memory required on a Pentaho server.</span></div>
</div>
</h2>
<div class="MsoNormal" style="tab-stops: 36.0pt; text-indent: 0cm;">
For further
details, please see MONDRIAN-1713.<o:p></o:p></div>
<div>
<div>
<br /></div>
<!--EndFragment--></div>
<br />Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com9tag:blogger.com,1999:blog-6644329693530300467.post-8329999491556076222019-07-15T12:30:00.007+01:002019-07-17T10:26:38.754+01:00Pentaho 8.3 is available!Sorry for the delay! The release was late last week but I was travelling. In all fairness, I could have done this friday night, but then again, it was friday night so... 🤪🍷<br />
<div>
<br /></div>
<div>
A note of thanks to the Pentaho PM team for the following copy-paste exercise!<br />
<br />
So let's start from the beginning and... <a href="https://community.hitachivantara.com/docs/DOC-1009931-downloads"><b>Download it here</b></a>!</div>
<div>
<br /></div>
<h2>
Pentaho 8.3</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAR1-ZYzlvF9gV2vIuGIyVwY_q1hkjc-GvZ7p_MdS1zKH4eI-EOUnbswkZDlE500cJZVQKsTu41uSsF0UK92771_iImk0oyuGf24L_1mobeGz08D4oZQNk9DeWNq6sgLKEdsmIQrV84h4/s1600/01_release_overview.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="643" data-original-width="1336" height="191" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAR1-ZYzlvF9gV2vIuGIyVwY_q1hkjc-GvZ7p_MdS1zKH4eI-EOUnbswkZDlE500cJZVQKsTu41uSsF0UK92771_iImk0oyuGf24L_1mobeGz08D4oZQNk9DeWNq6sgLKEdsmIQrV84h4/s400/01_release_overview.png" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div>
The major new product innovations and enhancements included in 8.3 are:</div>
<div>
<br /></div>
<div>
<ul>
<li>PDI Amazon Kinesis Streaming Integration</li>
<li>PDI Amazon Redshift Bulk Load</li>
<li>PDI + BA Snowflake Connectivity</li>
<li>PDI HCP (Hitachi Content Platform) Integration Enhancements</li>
<li>PDI SAP Connector</li>
<li>BA Viz API 3.0 General Availability</li>
</ul>
</div>
<div>
<br /></div>
<div>
Toward the end of the document, we have also briefly summarized other new capabilities and improvements that are included in 8.3. These additional features include:</div>
<div>
<ul>
<li>PDI AEL Enhancements</li>
<li>PDI + BA Upgrade Utility Enhancements</li>
<li>PDI VFS (Virtual File System) Updates</li>
<li>PDI Lineage Updates</li>
<li>PDI Metadata Injection Updates</li>
<li>PDI Python Executor Step Updates</li>
<li>BA Analyzer Improvements</li>
<li>BA Interactive Reporting Improvements</li>
</ul>
</div>
<div>
<br /></div>
<div>
For links to detailed product documentation on what is new in Pentaho 8.3, see here: <a href="https://help.pentaho.com/Documentation/8.3">https://help.pentaho.com/Documentation/8.3</a></div>
<div>
<br /></div>
<div>
An exhaustive list of additional product improvements and bug fixes are available in the 8.3 release notes, which can be found on the Support Portal: <a href="https://support.pentaho.com/hc/en-us">https://support.pentaho.com/hc/en-us</a></div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<h3>
PDI Amazon Kinesis Streaming Integration</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrc8CFcGEKC05Oej_v0Lv9S58OjniG4_LwE05CnANCGXHPDpKjaF0nF1FoIQRbWZaIoXm_lAkMhXIyiUlJuxT4UHmHbftNVTrTtvklKQ_EyxQAVoPw-58axsOvd8sorVvdb3bTIUsvCP4/s1600/02_kinesis.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="491" data-original-width="422" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrc8CFcGEKC05Oej_v0Lv9S58OjniG4_LwE05CnANCGXHPDpKjaF0nF1FoIQRbWZaIoXm_lAkMhXIyiUlJuxT4UHmHbftNVTrTtvklKQ_EyxQAVoPw-58axsOvd8sorVvdb3bTIUsvCP4/s320/02_kinesis.png" width="275" /></a></div>
<div>
<br /></div>
<h4>
Capability</h4>
<div>
Amazon Kinesis enables ingestion of real-time data such as video, audio, application logs, website clickstreams, and IoT telemetry data for machine learning, analytics, and other applications. Data can be processed as it becomes available, and Amazon Kinesis is real time, fully managed and scalable. It is important to note that Kinesis is an umbrella of multiple real time technologies, including Kinesis Data Streams, Kinesis Video Streams, Kinesis Firehose, and Kinesis Analytics.</div>
<div>
<br /></div>
<div>
Pentaho version 8.3 introduces supported PDI integration with AWS Kinesis Data Streams, including the following features:</div>
<div>
<br /></div>
<div>
<ul>
<li>Ability to ingest messages to PDI from AWS Kinesis (via new Kinesis Consumer transformation step) and output messages from PDI to AWS Kinesis (using new Kinesis Producer transformation step)</li>
<li>Micro-batch processing of data originating from AWS Kinesis, leveraging processing windows based on time periods (in milliseconds) or number of messages</li>
<li>Choice of executing micro-batch processing transformations with either the native Pentaho (Kettle) engine on Pentaho Server or the Spark Streaming engine on EMR Spark via Adaptive Execution Layer (AEL)</li>
<li>Support for Avro-typed read and write of data using Kinesis data streams</li>
<li>Support for max concurrent batches to allow transformation-specific scaling in Kettle engine. (Spark Streaming automatically handles this when executed in Spark via AEL).</li>
</ul>
</div>
<div>
<br /></div>
<h4>
Use Cases and Benefits</h4>
<div>
Kinesis provides real-time data capability in an AWS environment. Pentaho and Kinesis Data Streams can provide filtering and contextual analysis of streaming data even for very high-throughput streaming data sources. Use case examples include shop-floor monitoring, as well as predicting quality risk and recommending optimizations. Through AWS Kinesis and Pentaho stream processing integration, Pentaho enables AWS developers to ingest and process streaming data in a powerful visual environment.</div>
<h4>
Additional Considerations</h4>
<div>
Users should be aware of the following items related to stream data processing capability in Pentaho 8.3:</div>
<div>
<br /></div>
<div>
<ul>
<li>Kinesis steps are only available in Enterprise Edition (EE) version of Pentaho.</li>
<li>Kinesis Producer step doesn’t create a new Stream if one does not exist already.</li>
<li>Avro read/write and max concurrent batch capabilities are not supported for other streaming steps aside from the Kinesis steps.</li>
</ul>
</div>
<h4>
Additional Resources</h4>
<div>
Documentation on the Kinesis steps can be found below.</div>
<div>
<br /></div>
<div>
<ul>
<li>Consumer Step: <a href="https://help.pentaho.com/Documentation/8.3/Products/Kinesis_Consumer">https://help.pentaho.com/Documentation/8.3/Products/Kinesis_Consumer</a></li>
<li>Producer Step: <a href="https://help.pentaho.com/Documentation/8.3/Products/Kinesis_Producer">https://help.pentaho.com/Documentation/8.3/Products/Kinesis_Producer</a></li>
</ul>
</div>
</div>
<div>
<br /></div>
<div>
<h3>
PDI Amazon Redshift Bulk Load</h3>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhayO3id4d_ZCZGub7YyTLcQAPz_aT6OeEjZe1xwzhfL83etoBo4IuDQuOyWQHzCdeOKFhA4EsvKYpVQzRWUlLMoTYlcfDNK_Dmsn93kwAc6D2P6eyDKz3l6f12VQsiWE_SVklO4Gik6rI/s1600/03_bulk_loader.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="583" data-original-width="462" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhayO3id4d_ZCZGub7YyTLcQAPz_aT6OeEjZe1xwzhfL83etoBo4IuDQuOyWQHzCdeOKFhA4EsvKYpVQzRWUlLMoTYlcfDNK_Dmsn93kwAc6D2P6eyDKz3l6f12VQsiWE_SVklO4Gik6rI/s320/03_bulk_loader.png" width="253" /></a></div>
<div>
<br /></div>
<h4>
Capability</h4>
<div>
Redshift is a managed, highly scalable data warehouse service in the cloud that is offered by AWS, and Pentaho has provided integration with it for several years. PDI 8.3 includes a new job entry to visually configure bulk loads of files from AWS S3 storage to Redshift database tables. The entry orchestrates an AWS Redshift COPY Command and includes the following tabs:</div>
<div>
<br /></div>
<div>
<ul>
<li>Input Tab: Allows the user to select the S3 source for the bulk load (which can include multiple files), specify the format and compression of the source data, and configure commonly used properties relevant to the chosen format.</li>
<li>Output Tab: Lets the user specify the Redshift database connection, schema, and target table to load; also shows a preview of the table columns</li>
<li>Parameters Tab: User can input values for additional parameters for the COPY Command, related to error handling, field formats, and other items</li>
</ul>
</div>
<h4>
Use Cases and Benefits</h4>
<div>
Organizations bulk load data into Redshift to support business processes and operations such as:</div>
<div>
<br /></div>
<div>
<ul>
<li>Populating Redshift data warehouses for regular reporting needs under standard SLAs</li>
<li>Repetitive data onboarding with similar logic across many S3 sources and Redshift table targets</li>
<li>Cleansing and repackaging data for specific customers</li>
<li>Loading many data files at once in parallel for high performance vs other approaches</li>
</ul>
</div>
<div>
<br /></div>
<div>
The new bulk loader job entry helps PDI users avoid repetitive SQL scripting to orchestrate bulk loads. The GUI-based job entry takes advantage of high performance loading features of the COPY command, while supporting greater automation of onboarding to Redshift in PDI jobs.</div>
<h4>
Key Considerations</h4>
<div>
Users should be aware of the following when using the Redshift bulk load job entry in PDI:</div>
<div>
<ul>
<li>In conjunction with the introduction of the new entry, the Redshift database connection dialog now includes an S3 Authentication Method section that must be populated to ensure that Redshift has access to the proper AWS S3 data for bulk load</li>
<li>The new job entry can only be used to load Redshift from S3, and does not support loads from other sources (though PDI already enables moving data from other sources to S3)</li>
</ul>
</div>
<h4>
Additional Resources</h4>
<div>
For additional information on the PDI Redshift bulk load job entry, see here: <a href="https://help.pentaho.com/Documentation/8.3/Products/Bulk_load_into_Amazon_Redshift">https://help.pentaho.com/Documentation/8.3/Products/Bulk_load_into_Amazon_Redshift</a></div>
<div>
<br /></div>
<div>
For further information on the Redshift COPY Command parameters see here: <a href="https://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html">https://docs.aws.amazon.com/redshift/latest/dg/r_COPY.html</a></div>
<div>
<br /></div>
<h3>
PDI + BA Snowflake Database Connectivity</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRSqEQ8ijpVMyc5As9d013UsNwaYTjatWC9mSYiCWDKahXDlpVlc0GdYx2GZmd6yuf3FhCR0FzN6elgJwXYVWKPTuYjtriXjxQzga9t4LtHMhY4MpE1eObJx7JIOR4QaAj5SzJUZvQeOA/s1600/04_snowflake.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="451" data-original-width="559" height="257" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgRSqEQ8ijpVMyc5As9d013UsNwaYTjatWC9mSYiCWDKahXDlpVlc0GdYx2GZmd6yuf3FhCR0FzN6elgJwXYVWKPTuYjtriXjxQzga9t4LtHMhY4MpE1eObJx7JIOR4QaAj5SzJUZvQeOA/s320/04_snowflake.png" width="320" /></a></div>
<div>
<br /></div>
<h4>
Capability</h4>
<div>
Snowflake is a cloud-based data warehouse that speaks SQL and combines flexible compute clusters for query processing with a central data repository. Snowflake customers can scale compute and storage independent of each other as they manage data, and the resulting costs are also incurred independently.</div>
<div>
<br /></div>
<div>
Pentaho 8.3 includes supported database connectivity to Snowflake across the platform, including Pentaho Data Integration, Pentaho User Console, Analyzer, Interactive Reporting, Report Designer, Schema Workbench, and Metadata Editor.</div>
<div>
<br /></div>
<div>
A specific ‘Snowflake’ database connection type will now be available in these tools, and optimized SQL generation and dedicated dialects to work with Snowflake were added for Pentaho Data Integration, Analyzer (Mondrian), and Reporting (Metadata).</div>
<h4>
Use Cases and Benefits</h4>
<div>
The 8.3 connectivity enables blending, enrichment, and analysis of Snowflake data along with other data sources. It also helps to support customers in their ongoing journey toward cloud and hybrid environments for data management and analytics.</div>
<div>
<br /></div>
<div>
More specifically, customers who use the end-to-end Pentaho platform with on-premise analytic databases, and are planning migrations to Snowflake, can now leverage the different Pentaho tools with Snowflake for data orchestration, reporting, ad hoc analysis, and other use cases. A “lift and shift” approach like this is a popular first step when going to the cloud to take advantage of costs savings, elasticity, and other business benefits without drastically changing data management architecture and processes.</div>
<div>
<br /></div>
<div>
It is also worth noting that Pentaho equips customers to leverage data across multiple cloud platforms, including AWS and Google Cloud, in addition to Snowflake.</div>
<h4>
Key Considerations</h4>
<div>
Customers should be aware of the following points as they begin to use the Pentaho connectivity to Snowflake in 8.3:</div>
<div>
<ul>
<li>8.3 ships with Snowflake JDBC drivers for Pentaho Server and PDI, however, for other client tools the drivers may need to be manually copied in the proper locations – see here for more detail: <a href="https://help.pentaho.com/Documentation/8.3/Setup/Install_drivers_with_the_JDBC_distribution_tool">https://help.pentaho.com/Documentation/8.3/Setup/Install_drivers_with_the_JDBC_distribution_tool</a></li>
<li>When loading Snowflake with PDI 8.3, it is advised to orchestrate a SQL COPY INTO command rather than using the Table Output step due to slow Snowflake performance with insert statements</li>
<ul>
<li>Note -- this process should be improved in the near future, as Pentaho anticipates releasing a PDI Bulk Load capability and other Snowflake integration enhancements later in 2019</li>
</ul>
<li>As of 8.3 release, Aggregation Designer is not supported for Snowflake connectivity</li>
</ul>
</div>
<h4>
Additional Resources</h4>
<div>
Information regarding the Snowflake JDBC driver to be used with Pentaho can be found here: <a href="https://help.pentaho.com/Documentation/8.3/Setup/JDBC_drivers_reference#Snowflake">https://help.pentaho.com/Documentation/8.3/Setup/JDBC_drivers_reference#Snowflake</a></div>
<div>
<br /></div>
<div>
Learn more about Snowflake here: <a href="https://docs.snowflake.net/manuals/index.html">https://docs.snowflake.net/manuals/index.html</a></div>
<div>
<br /></div>
<h3>
PDI HCP (Hitachi Content Platform) Integration Enhancements</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKXAo2m_AkT6mnu5oaiaUzgRfoFdd_Z31pNMTsaNXf-2kBJ8Xz57k6qplFZZYFxyP5AvgA904qSvgBrIkRaXumw_P9MpvCBm85yaViz7klVPqS3Psayojyz8deAo0k7-am-ULoNjAuVUo/s1600/05_HCP_Metadata.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="561" data-original-width="919" height="195" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgKXAo2m_AkT6mnu5oaiaUzgRfoFdd_Z31pNMTsaNXf-2kBJ8Xz57k6qplFZZYFxyP5AvgA904qSvgBrIkRaXumw_P9MpvCBm85yaViz7klVPqS3Psayojyz8deAo0k7-am-ULoNjAuVUo/s320/05_HCP_Metadata.png" width="320" /></a></div>
<div>
<br /></div>
<h4>
Capability</h4>
<div>
Since version 8.2 PDI has been able to connect to Hitachi Content Platform (HCP) via the Virtual File System (VFS) to read, write, update or delete data. In 8.3, it is now possible to read, write and update HCP custom metadata and to query objects with their system metadata.</div>
<div>
<br /></div>
<div>
HCP is a distributed object storage system designed to support large, growing repositories of fixedcontent data from simple text files to images and video to multi-gigabyte database images. HCP can be used as a Globally Compliant Retention Platform (GCRP) that meets compliance & legal retention requirements (WORM, SEC 17A-4, CFTC and MSRB). It can also be used as a secure analytics archive. HCP protects data with very high durability (up to fifteen 9s) and availability (up to ten 9s).</div>
<div>
<br /></div>
<div>
Each HCP object permanently associates data HCP receives (for example, a document, an image, or a movie) with information about that data called metadata. In PDI, you can query the metadata to locate and access HCP objects. The HCP object consists of the data (such as an image file), a unique URL to it, system metadata properties, and custom metadata annotations.</div>
<div>
<br /></div>
<div>
In 8.3, PDI has three transformation steps you can use to work with your metadata in HCP:</div>
<div>
<br /></div>
<div>
The Query HCP step locates data objects by searching system and custom metadata annotations. HCP returns the unique URL and system metadata of objects matching your search terms. For example, a radiology practice can search for an X-ray (object) of a specific patient or all X-rays performed by a specified physician.</div>
<div>
<br /></div>
<div>
With the Read metadata from HCP step, you can identify and select an HCP object by its URL path and then select a specific target annotation name to read. The step returns the requested custom metadata from the annotation back to your PDI transformation for downstream processing.</div>
<div>
<br /></div>
<div>
With the Write metadata to HCP step, you can identify and select an HCP object by its URL and then write custom metadata annotations to the object associated with the object URL, enriching and validating the data stored in your HCP repository. For example, a radiology practice could add patient medication data and associated medical conditions to an X-ray (object) or remove invalid diagnosis codes.</div>
<h4>
Use Cases and Benefits</h4>
<div>
HCP + PDI can act as a “Staging Data Lake” for semi-structured and unstructured data. PDI provides data quality, enrichment and cleaning before moving to an enterprise data warehouse or big data lake. You can execute data science workflows (for example deep learning for image recognition) against structured/unstructured data in HCP as well.</div>
<div>
<br /></div>
<div>
Nearly every PDI step and job entry that references files can use VFS and now also can access data on HCP and starting in 8.3 can add, read and modify custom metadata and query objects.</div>
<h4>
Key Considerations</h4>
<div>
In this release we support HCP versions 8.0 and 8.1. There is no support for HCP Anywhere nor HCP Cloud Scale.</div>
<h4>
Additional Resources</h4>
<div>
Please see the documentation of PDI steps that work with HCP: <a href="https://help.pentaho.com/Documentation/8.3/Products/PDI_and_Hitachi_Content_Platform_(HCP)">https://help.pentaho.com/Documentation/8.3/Products/PDI_and_Hitachi_Content_Platform_(HCP)</a></div>
<div>
<br /></div>
<div>
VFS Documentation with HCP Details: <a href="https://help.pentaho.com/Documentation/8.3/Products/Virtual_file_system_browser">https://help.pentaho.com/Documentation/8.3/Products/Virtual_file_system_browser</a></div>
<div>
<br /></div>
<div>
HCP Documentation: <a href="https://knowledge.hitachivantara.com/Documents/Storage/Content_Platform">https://knowledge.hitachivantara.com/Documents/Storage/Content_Platform</a></div>
<div>
<br /></div>
<h3>
PDI SAP Connector</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGXdcumP3vqdByCuU-FMJWnBTaUSLz4iwylTiYpEoC0vr3Uijhh3WyE_SD1ZTFi_5VLKB9kesBoQUqcqFRR72ZsaiQWGhsRBBhGMtVkgtoUGgsUr7LCcNqgLuwfNevjaq-oVGLNzi6NhE/s1600/06_SAP.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="471" data-original-width="807" height="186" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgGXdcumP3vqdByCuU-FMJWnBTaUSLz4iwylTiYpEoC0vr3Uijhh3WyE_SD1ZTFi_5VLKB9kesBoQUqcqFRR72ZsaiQWGhsRBBhGMtVkgtoUGgsUr7LCcNqgLuwfNevjaq-oVGLNzi6NhE/s320/06_SAP.png" width="320" /></a></div>
<div>
<br /></div>
<h4>
Capability</h4>
<div>
Note – the steps comprising the SAP Connector have been built by partner IT-Novum and are supported jointly by Hitachi Vantara and IT-Novum. They were first made available between the 8.2 and 8.3 releases of Pentaho. The SAP Connector is available and priced separately from the core Pentaho platform. Please connect with your sales representative for further details.</div>
<div>
<br /></div>
<div>
The Hitachi Data Connector for SAP ERP & Business Warehouse (BW) has the following capabilities:</div>
<div>
<br /></div>
<div>
<ul>
<li>Flexible and easy to use analysis of SAP data</li>
<li>Query of complex and nested SAP structures (such as cost center groups or structures/data only available at runtime of the SAP system)</li>
<li>Access to BW data using SAP Data Store Objects</li>
<li>Support for Metadata Injection</li>
<li>Cost-saving offload scenarios</li>
<li>Data Blending of SAP Data</li>
<li>Analytical Workloads on Data Warehouse Layer (SAP Infocubes)</li>
<li>High performance data transfer from SAP to Hadoop thanks to integrated server mode</li>
</ul>
</div>
<div>
<br /></div>
<div>
The connector contains the following PDI steps:</div>
<div>
<br /></div>
<div>
<ul>
<li>The SAP ERP Table Input step allows you to load data from SAP tables for further processing.</li>
<li>The SAP BW/ERP RFC Executor step calls remote functions and supports SAP Business Warehouse and ERP and allows to solve special use cases by executing RFC/BAPIs (Remote Function Calls / Business Application Programming Interface).</li>
<li>The SAP BW DSO Input (Data Store Objects) step is capable of loading data from SAP BW – Data Store Objects. It also supports advanced Data Store Objects (A-DSO) with SAP HANA.</li>
</ul>
</div>
<div>
<br /></div>
<div>
For reference, PDI separately offers support for integration to SAP HANA in PDI as a database connection (since PDI 5.4) and ability to bulk load data to SAP HANA (since PDI 6.0).</div>
<h4>
Use Cases and Benefits</h4>
<div>
With the Connector, application scenarios and use cases such as onboarding, blending or offloading of SAP ERP and SAP Business Warehouse data can easily be realized.</div>
<div>
<br /></div>
<div>
Furthermore, the steps have extended capabilities that offer full flexibility such as data retrieval and connection to SAP Business Processes (BAPI), calling many existing SAP functions, and write-back into SAP.</div>
<div>
<br /></div>
<div>
Pentaho's PDI enables companies to easily connect data from SAP ERP, SAP BW, and HANA with data from non-SAP systems to create powerful analytics applications.</div>
<div>
<br /></div>
<div>
The connector also helps to secure investments that companies have already made in SAP components such as HANA Views & Tables, Bex Queries, DSO / ODS objects, ABAP reports, SAP queries and SAP extractors.</div>
<div>
<br /></div>
<h4>
Key Considerations</h4>
<div>
The SAP Connector is available and priced separately from the core Pentaho platform. Please connect with your sales representative for further details.</div>
<div>
<br /></div>
<div>
The Connector is a separate plugin and supports PDI Version 7.x and 8.x.</div>
<h4>
Additional Resources</h4>
<div>
SAP and Analytics: <a href="https://it-novum.com/en/big-data-analytics/sap-and-analytics/">https://it-novum.com/en/big-data-analytics/sap-and-analytics/</a></div>
<div>
<br /></div>
<div>
Hitachi Data Connector SAP ERP & BW: <a href="https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fit-novum.com%2Fen%2Fbig-data-analytics%2Fsap-and-analytics%2Fhitachi-data-connector-sap-erp-bw%2F&data=01%7C01%7Cpedro.alves%40hitachivantara.com%7C85d69add706944a5a83808d70a98b455%7C18791e1761594f52a8d4de814ca8284a%7C0&sdata=mP9IJjHp44UQlOWWJ4h9J7FPEkB1oUvRLmSBGysukPM%3D&reserved=0" originalsrc="https://it-novum.com/en/big-data-analytics/sap-and-analytics/hitachi-data-connector-sap-erp-bw/" shash="mItcT/4yOZZHA7g51xZwETu5FGa27kOGToJLPXCff7Cm/tZflg4AiI6Q9HKeAh8+BxTs3d/7NFSf5bIPxKTXW6/v5USOqXw3wXh3M1C8/C+gx4IZWHYNmvVvjjQujaZb0hJoxLyntGrkh/LVH6auxpeIJIzEwHZ07/qqUmasFbs=" style="color: #954f72; font-family: Calibri, sans-serif; font-size: 14.6667px; text-size-adjust: auto;" title="Original URL:
https://it-novum.com/en/big-data-analytics/sap-and-analytics/hitachi-data-connector-sap-erp-bw/
Click to follow link.">https://it-novum.com/en/big-data-analytics/sap-and-analytics/hitachi-data-connector-sap-erp-bw/</a></div>
<div>
<br /></div>
<h3>
BA Viz API 3.0 General Availability</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjs8X3XgP2aTdUmjlBSiKQbl47fZf6XnjLu-phyphenhyphenUYUmMy1x03sXHvcnxT5LeUJJkyoSZ4dRHMRWUpZEpre-j33RKlmzItmbN7U-n1EkQnkkgw4h3_0egYur_8UDdTE9OpAC9JQcPjAzZgU/s1600/07_vizapi.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="283" data-original-width="782" height="115" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjs8X3XgP2aTdUmjlBSiKQbl47fZf6XnjLu-phyphenhyphenUYUmMy1x03sXHvcnxT5LeUJJkyoSZ4dRHMRWUpZEpre-j33RKlmzItmbN7U-n1EkQnkkgw4h3_0egYur_8UDdTE9OpAC9JQcPjAzZgU/s320/07_vizapi.png" width="320" /></a></div>
<div>
<br /></div>
<h4>
Capability</h4>
<div>
The Visualization API 3.0 is part of Pentaho’s infrastructure that enables Analyzer, CTools, and Data Explorer (in PDI) to use visualizations in a unified, pluggable way. Further, it provides a simple, powerful, tested, and documented approach for developers to integrate new visualizations to use in these tools and configure properties of visualizations used in them.</div>
<div>
<br /></div>
<div>
Viz API was first introduced as an evolution from Viz API 2.0 in Pentaho version 7.1, however, it has technically been in a ‘beta’ state (such that it could continue to be improved) until now – in Pentaho 8.3 the Viz API 3.0 is generally available and supported. This means that Pentaho provides support for customer usage of the API, as documented, in order to integrate and configure visualizations. Further, customers can be confident that changes to the API will be carefully managed and communicated going forward.</div>
<div>
<br /></div>
<div>
Viz API allows Pentaho tools to leverage visualization libraries and configurations in a unified way</div>
<h4>
Use Cases and Benefits</h4>
<div>
Viz API 3.0 makes it easier for developers to integrate 3 rd party visualizations and develop new charts for use in Pentaho. This can be achieved by adapting code from 3 rd party libraries (such as D3) to be used in Pentaho per documentation provided. Further, Viz API is built on top of other Platform JavaScript APIs that ensure seamless data access and provide visualizations with validation, theming, and other features.</div>
<div>
<br /></div>
<div>
In addition, Viz API 3.0 provides the ability to reuse configurations across Analyzer, CTools, and Data Explorer (PDI).</div>
<h4>
Key Considerations</h4>
<div>
The following information should be carefully considered for customers planning to migrate from Viz 2.0 to Viz 3.0.</div>
<div>
<br /></div>
<div>
<ul>
<li>Please Note – to successfully convert reports from Viz API 2.0 to Viz API 3.0 upon upgrade from an early Pentaho version, some customers may first need to make additional configuration file changes to 8.3. Please contact Support for additional details.</li>
<li>Viz 2.0 is the default Viz API for customers that upgrade from a previous version of Pentaho and using Viz 2.0 with that version. Viz 2.0 continues to be supported in Pentaho 8.3. However, stock (out of box) visualizations can be converted from 2.0 to 3.0.</li>
<li>The Viz API version for Analyzer can be switched by the administrator in the Analyzer settings.xml configuration file, and after the switch is made, the conversion to Viz 3.0 for any given report is triggered by saving that report.</li>
<li>Before migrating to Viz 3.0, it is advised that customers back up their existing Analyzer reports and then test them with Viz API 3.0 before saving any report.</li>
<li>3.0 visualizations are configured in a global configuration file vs. 2.0 configuration in an Analyzerspecific config file. The syntax used for 3.0 syntax is also different than the 2.0 syntax.</li>
<li>Custom visualizations created with Viz 2.0 will always display as Viz 2.0 and cannot be converted to viz 3.0 via configuration.</li>
</ul>
</div>
<h4>
Additional Resources</h4>
<div>
Documentation on Viz API 3.0 can be found at the following links --</div>
<div>
<br /></div>
<div>
<ul>
<li>Overview of Viz API 3.0: <a href="https://help.pentaho.com/Documentation/8.3/Developer_center/Visualization_API">https://help.pentaho.com/Documentation/8.3/Developer_center/Visualization_API</a></li>
<li>Analyzer Considerations: <a href="https://help.pentaho.com/Documentation/8.3/Developer_center/Analyzer_and_the_Visualization_API">https://help.pentaho.com/Documentation/8.3/Developer_center/Analyzer_and_the_Visualization_API</a></li>
<li>Configuration: <a href="https://help.pentaho.com/Documentation/8.3/Developer_center/Configuring_a_visualization">https://help.pentaho.com/Documentation/8.3/Developer_center/Configuring_a_visualization</a></li>
</ul>
</div>
<div>
<br /></div>
<h3>
Additional Enhancements</h3>
<div>
The following capabilities have also been introduced in the Pentaho platform as part of the 8.3 release.</div>
<h4>
PDI AEL Enhancements</h4>
<div>
The Pentaho Adaptive Execution Layer (AEL) is intended to provide flexible, transparent data processing with Spark in addition to the native Kettle engine. The goal of AEL is to develop complex pipelines visually and then execute in Kettle or Spark based on data volume and SLA requirements. AEL allows PDI users to designate Spark as execution engine for their transformations apart from Kettle.</div>
<div>
<br /></div>
<div>
The 8.3 release includes the following performance and flexibility enhancements to AEL-Spark:</div>
<div>
<ul>
<li>Spark Technology update -- Upgrading underlying data structure from RDD to Dataset</li>
<li>Adding native Spark execution for additional PDI transformation steps such as Switch-case and Merge Rows</li>
<li>Upgraded Hadoop distribution version support for AEL</li>
</ul>
</div>
<div>
<br /></div>
<div>
Users should be aware of the following additional items related to AEL 8.3:</div>
<div>
<ul>
<li>Spark native Hive steps only work with HDP distributions below 3.0.</li>
<li>Native HBase steps are only available for CDH and HDP distributions.</li>
<li>Spark 2.3 is the highest Spark version currently supported.</li>
<li>CDH 6.0 and higher versions are not supported as these versions only package Spark 2.4.</li>
</ul>
</div>
<h4>
PDI + BA Upgrade Utility Enhancements</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiea24mFO66uHUSMPxo1fxAgacfE32YrwnmTRJCuNwDLFCCX4bJTQVwScd_6MZzBk_Q3Qr8K2lxjkQCByfAjZq6mkMCtFuTR24_e9Fcj_hTtx2mnFm-0rzgwrqtL-7oUU6pEhbc6ietzIQ/s1600/08_upgrade_utility.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="245" data-original-width="560" height="140" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiea24mFO66uHUSMPxo1fxAgacfE32YrwnmTRJCuNwDLFCCX4bJTQVwScd_6MZzBk_Q3Qr8K2lxjkQCByfAjZq6mkMCtFuTR24_e9Fcj_hTtx2mnFm-0rzgwrqtL-7oUU6pEhbc6ietzIQ/s320/08_upgrade_utility.png" width="320" /></a></div>
<div>
<br /></div>
<div>
In Version 8.2, we introduced the Pentaho Server Upgrade Installer that we continue to deliver in 8.3 for further upgrade paths. It is an easy to use graphical user interface that automatically applies the minor release version to your existing Pentaho installation.</div>
<div>
<br /></div>
<div>
Pentaho versions 8.1 and 8.2 can be upgraded directly to Pentaho 8.3. The installer can also be invoked via command line for automated deployment scenarios. For all other upgrade paths from versions prior to 8.1, the pre-existing manual upgrade process is still the supported method of upgrade.</div>
<div>
<br /></div>
<div>
Further information about the upgrade installer and the download can be found in the customer support portal: <a href="https://support.pentaho.com/hc/en-us/categories/Downloads">https://support.pentaho.com/hc/en-us/categories/Downloads</a></div>
<div>
<br /></div>
<div>
General upgrade instructions can be found in the following documentation: <a href="https://help.pentaho.com/Documentation/8.3/Setup/Pentaho_upgrade">https://help.pentaho.com/Documentation/8.3/Setup/Pentaho_upgrade</a></div>
<h4>
PDI VFS (Virtual File System) Updates</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkqgwNbJBITQ5I_lsuzeUcGb1bQ1xMWUupSD-AFByPNPfHI9lW3Rid1DhQP52j8GuahgkK5JuWs_RJuzIto4U4VoEh2_h24v60xKAS4gwxwMUViqPVBcB5t73_-8xCjtLY4yDW4R8dpJY/s1600/09_vfs_connector.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="445" data-original-width="435" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjkqgwNbJBITQ5I_lsuzeUcGb1bQ1xMWUupSD-AFByPNPfHI9lW3Rid1DhQP52j8GuahgkK5JuWs_RJuzIto4U4VoEh2_h24v60xKAS4gwxwMUViqPVBcB5t73_-8xCjtLY4yDW4R8dpJY/s320/09_vfs_connector.png" width="312" /></a></div>
<div>
<br /></div>
<div>
In 8.3 we started to unify the VFS property settings for different VFS providers. In this release we unified the settings for the HCP and Google Cloud Storage VFS providers in order to improve overall usability. Named VFS connections that can be reused for multiple steps and job entries, can be easily modified and used in different environments, and most importantly, there is a single place to set VFS properties. In general, we plan to apply the settings unification to other VFS providers, such as Amazon and others, in future releases.</div>
<h4>
PDI Lineage Updates</h4>
<div>
In this release, we added custom metaverse analyzers for data lineage tracking with respect to the following transformation steps:</div>
<div>
<ul>
<li>AMQP Consumer & Producer</li>
<li>JMS Consumer & Producer</li>
<li>Kafka Consumer & Producer</li>
<li>MQTT Consumer & Producer</li>
</ul>
</div>
<div>
This expands the completeness of data lineage for PDI.</div>
<div>
<br /></div>
<div>
To view the full list of steps and entries with custom data lineage analyzers, see Data Lineage:</div>
<div>
<br /></div>
<div>
https://help.pentaho.com/Documentation/8.3/Products/Data_lineage</div>
<div>
<br /></div>
<div>
Additionally, the following features related to data lineage have been added as beta features that are not currently supported:</div>
<div>
<br /></div>
<div>
<ul>
<li>Support for IBM IGC (Beta) enables PDI lineage to be integrated with an existing governance solution IBM IGC (Information Governance Catalog)</li>
<li>The Spark lineage feature (Beta) can get run time data lineage for AEL-Spark run jobs.</li>
</ul>
</div>
<div>
<br /></div>
<div>
If you are interested in the above beta features, please contact support or your sales representative.</div>
<h4>
PDI Metadata Injection Updates</h4>
<div>
Metadata injection enables the passage of metadata to transformation templates at runtime to drastically increase productivity, reusability, and automation of transformation workflow. This supports use cases like the onboarding of data from many files and tables to data lakes. In addition to existing metadata injection enabled steps, as of 8.3 you now can inject metadata into any field in the following Pentaho Data Integration (PDI) steps:</div>
<div>
<ul>
<li>Table Output (added Connection Field)</li>
<li>Strings cut</li>
<li>Salesforce Input</li>
</ul>
</div>
<div>
Learn more about PDI steps supporting metadata injection: <a href="https://help.pentaho.com/Documentation/8.3/Products/ETL_metadata_injection">https://help.pentaho.com/Documentation/8.3/Products/ETL_metadata_injection</a></div>
<h4>
PDI Python Executor Step Updates</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG9zF5ZKvPHvDM4QvEOiPDEi0G5k3lw2oQTuNGcjCZ3pyv-hagRVQNy-JRR_7y4DYPEsWCp419L5kq4hwQxjb3bO4FQ5G4KRcVhO4IQRVQMdu6Yx_nmSWDiZc6ju2YNDcXI9RzJfEKvYw/s1600/10_python.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="613" data-original-width="641" height="306" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgG9zF5ZKvPHvDM4QvEOiPDEi0G5k3lw2oQTuNGcjCZ3pyv-hagRVQNy-JRR_7y4DYPEsWCp419L5kq4hwQxjb3bO4FQ5G4KRcVhO4IQRVQMdu6Yx_nmSWDiZc6ju2YNDcXI9RzJfEKvYw/s320/10_python.png" width="320" /></a></div>
<div>
<br /></div>
<div>
The Python Executor step now features a new Python Data Structure, in addition to the existing Pandas dataframe and NumPy Array selections, named Python List of Dictionaries. The Data structure drop down menu is where you select the List of dictionaries data structure. In addition, the Python.org version that the Python Executor step is compatible with has been updated to include Python 3.7.x. Overall, these enhancements allow for more Python data structure choices and strive to keep compatibility more current for improved usability.</div>
<h4>
BA Analyzer Improvements</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUYaDmgHf7qIEOa9Aie0Q8rrFcLRSMDVIkFQLaSlInxaeL64Q6WCjZ03hwwk2_lwazKVW0KGxvZBswdOA6buYTEWTkV7i7xU2Z-P9GvYUVMhZKczDo2yO9nwUY367XVvgUQy1wfD8929Q/s1600/11_analyzer_excel.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="312" data-original-width="911" height="109" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiUYaDmgHf7qIEOa9Aie0Q8rrFcLRSMDVIkFQLaSlInxaeL64Q6WCjZ03hwwk2_lwazKVW0KGxvZBswdOA6buYTEWTkV7i7xU2Z-P9GvYUVMhZKczDo2yO9nwUY367XVvgUQy1wfD8929Q/s320/11_analyzer_excel.png" width="320" /></a></div>
<div>
<br /></div>
<div>
In Pentaho 8.3, there were several improvements related to Analyzer export experience and capabilities. They include:</div>
<div>
<br /></div>
<div>
<ul>
<li>Un-merge Analyzer cells on Export to Excel: There is now a ‘merge pivot table cells’ checkbox in the Export to Excel flow. When unchecked, the resulting Excel spreadsheet exported will unmerge cells that were merged in the Analyzer online view, such as label cells for higher levels in a hierarchy and label cells across multiple columns. Ultimately, this makes data manipulation of the exported data easier for Excel users. (This update addresses the request in Analyzer-2055 from JIRA)</li>
<li>Change CSV separator for Analyzer Export: In the Export to CSV flow, there is now a ‘Separator’ input box where the user can specify what separator they want to include in the export. This helps to streamline use cases where a separator other than comma is needed, as may come up in regions where comma is normally used as a decimal point. (This update addresses the request in Analyzer-516 from JIRA)</li>
<li>Export Analyzer report via REST API Call: There is a new supported REST endpoint to export Analyzer reports to PDF, CSV, and Excel. It can also be involved via URL, and is useful for embedded analytics use cases. Developer documentation is forthcoming on this enhancement. (This update addresses the requests in Analyzer-867 and Analyzer-2967 in JIRA)</li>
</ul>
</div>
<h4>
BA Interactive Reporting Improvements</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5q3BpgAUi9r6S4J-9I40pOSr2Pft_JNmK5ht6BZHa5OWLoljBgva-hdHiLAe7XiaLB5y5Wo6O7MqBbBP86G2ZViphxojtnOb7uuQwKHxZWFCk8W1lQISuA4ZIOblyY6I_mt6lXjdHyl0/s1600/12_pir.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="426" data-original-width="718" height="189" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5q3BpgAUi9r6S4J-9I40pOSr2Pft_JNmK5ht6BZHa5OWLoljBgva-hdHiLAe7XiaLB5y5Wo6O7MqBbBP86G2ZViphxojtnOb7uuQwKHxZWFCk8W1lQISuA4ZIOblyY6I_mt6lXjdHyl0/s320/12_pir.png" width="320" /></a></div>
<div>
<br /></div>
<div>
In Business Analytics, there were two notable continuous improvements related to Pentaho Interactive Reporting in 8.3. These enhancements are:</div>
<div>
<br /></div>
<div>
<ul>
<li>Search Fields: Users of Interactive Reporting can now find fields they are look for by using a text search box at the top of the Data pane. This accelerates user productivity in scenarios where data sources have many fields. (This update addresses the request from PIR-875 in JIRA)</li>
<li>Disable/Enable Select Distinct Setting: Administrators can now set the ‘Select Distinct’ option in Interactive Reporting Query Settings as disabled or enabled by default for users. This is done with in the settings.xml configuration file for Interactive Reporting. The enhancement helps to optimize performance and efficiency of queries across users in the Pentaho deployment. (This update addresses the request from PIR-760 in JIRA)</li>
</ul>
</div>
<h4>
Additional Resources</h4>
<div>
<br /></div>
<div>
These resources include additional useful information regarding Pentaho 8.3:</div>
<div>
<br /></div>
<div>
<ul>
<li>• Detailed Documentation: In-depth documentation on Pentaho 8.3 can be found here: <a href="https://help.pentaho.com/Documentation/8.3">https://help.pentaho.com/Documentation/8.3</a></li>
<li>Upgrade Information: Documentation on upgrading to Pentaho 8.3 from previous Pentaho versions can be found here: <a href="https://help.pentaho.com/Documentation/8.3/Setup/Pentaho_upgrade">https://help.pentaho.com/Documentation/8.3/Setup/Pentaho_upgrade</a></li>
<li>Compatibility Update: Details about support for Pentaho 8.3 with different technology components and versions can be found here: <a href="https://help.pentaho.com/Documentation/8.3/Setup/Components_Reference">https://help.pentaho.com/Documentation/8.3/Setup/Components_Reference</a></li>
<li>Release Notes: Details on which outstanding bugs and feature requests were resolved in Pentaho version 8.3 can be found on the Support Portal: <a href="https://support.pentaho.com/hc/en-us">https://support.pentaho.com/hc/en-us</a></li>
</ul>
</div>
</div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com4tag:blogger.com,1999:blog-6644329693530300467.post-30815451456393688442018-12-04T14:08:00.002+00:002018-12-04T18:07:58.679+00:00Pentaho 8.2 is available!<br />
I've come to accept my inefficiency on keeping up with the technical blog posts. This is the point where one accepts his complete uselessness (and I don't even know if this is a real word!)<br />
<br />
Anyway - up to the good things:<br />
<br />
<h2>
Pentaho 8.2 is available!</h2>
<div>
<br /></div>
<div>
<a href="https://community.hitachivantara.com/docs/DOC-1009931-downloads">Get it here!</a></div>
<div>
<br /></div>
<div>
A really really solid release! A huge list of things that will make a serious impact on the development effort and production releases out there.</div>
<div>
<br /></div>
<h3>
Release overview</h3>
<div>
Here's the release at a glimpse:</div>
<div>
<ul>
<li>Enhance Eco System Integration</li>
<ul>
<li>Hitachi Content Platform (HCP) Connector I</li>
<li>MapR DB Support</li>
<li>Google Encryption Support</li>
</ul>
<li>Improve Edge to Cloud Processing</li>
<ul>
<li>Enhanced AEL </li>
<li>Streaming AMQP</li>
</ul>
<li>Better Data Operation</li>
<ul>
<li>Expanded Lineage </li>
<li>Status Monitoring UX</li>
<li>OpenJDK support</li>
</ul>
<li>Enable Data Science & Visualization</li>
<ul>
<li>Python Executor</li>
<li>PDI Data Science Notebook (Jupyter) Integration</li>
<li>Push Streaming</li>
</ul>
<li>Improve Platform Stability and Usability</li>
<ul>
<li>JSON Enhancements</li>
<li>BA Chinese Language Localization for PUC</li>
<li>Expanded MDI</li>
</ul>
<li>Additional Improvements</li>
</ul>
</div>
<div>
<br /></div>
<div>
And now a little bit of detail into each of them:</div>
<div>
<br /></div>
<h3>
Ecosystem Integration</h3>
<h4>
Hitachi Content Platform (HCP) Connectivity</h4>
<div>
<div>
HCP is a distributed storage system designed to support large, growing repositories of fixed-content data from simple text files to images, video to multi-gigabyte database images. HCP stores objects that include both data and metadata that describes that data and presents these objects as files in a standard directory structure.</div>
<div>
<br /></div>
<div>
An HCP repository is partitioned into namespaces owned and managed by tenants, providing access to objects through a variety of industry-standard protocols, as well as through various HCP-specific interfaces.</div>
</div>
<div>
<br /></div>
<div>
There are many use cases for using HCP in the Enterprise context:</div>
<div>
<br /></div>
<div>
<ul>
<li>Globally Compliant Retention Platform (GCRP)</li>
<ul>
<li>Meet Compliance & Legal Retention requirements (WORM, SEC 17A-4, CFTC and MSRB)</li>
</ul>
<li>Secure Analytics Archive</li>
<ul>
<li>Big data source/target (land) for secure analytic workflows</li>
<li>Better Data portability</li>
<li>Multi-tenant</li>
</ul>
<li>Protect data with much higher durability (up to fifteen 9s) and availability (up to ten 9s) with HCP</li>
</ul>
<div>
<br /></div>
</div>
<div>
The PDI+HCP combo will allow much more resources into serving these use cases: By leveraging PDI's connectivity capabilities to a wide variety of data, we can use HCP as a "Staging Data Lake" for semi-structured and unstructured data and/or using it as an execution environment for the execution of data science algorithms against this type of content will also, like enriching HCP metadata or doing deep learning for image recognition</div>
<div>
<br /></div>
<div>
In this release we implemented a VFS driver for HCP; Next versions will include a deeper, metadata level integration with HCP's functionality.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4s0gRVnC4r8mfmm1tdVyVHU-dNimBXRw42SxnsaBxMQaAQcd3TwNNRcQSIICpXji8t0SL1onP_EKpWQYBcJFRzl1LK7HoqMgGI866kE8S2dLrBndA2UEi7CUOEleceKLmF8iFw9KWX5g/s1600/hcp_1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1126" data-original-width="906" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg4s0gRVnC4r8mfmm1tdVyVHU-dNimBXRw42SxnsaBxMQaAQcd3TwNNRcQSIICpXji8t0SL1onP_EKpWQYBcJFRzl1LK7HoqMgGI866kE8S2dLrBndA2UEi7CUOEleceKLmF8iFw9KWX5g/s320/hcp_1.png" width="257" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
MapR DB support</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMYdR3jLcJq62sVptCVVFCypK0-x07vJP0xz9FTh17g_32mtDKoO8grhAc-s8dil5PHO3KCTeg7AnTP6RaZ87_TRQrxxIgFGsDGZO0otBRphLGyREXgATLx5pu-7CYNgZbObOKZwce65k/s1600/mapr.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="531" data-original-width="600" height="283" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjMYdR3jLcJq62sVptCVVFCypK0-x07vJP0xz9FTh17g_32mtDKoO8grhAc-s8dil5PHO3KCTeg7AnTP6RaZ87_TRQrxxIgFGsDGZO0otBRphLGyREXgATLx5pu-7CYNgZbObOKZwce65k/s320/mapr.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Simple but important improvement: MapR DB is now supported! It's an enterprise-grade, high performance, global NoSQL database management system. It is a multi-model database that converges operations and analytics in real-time, including the HBase API to run HBase applications, even though not all features are compatible.</div>
<div>
<br /></div>
<div>
It's now validated to read/write data from MapR-DB as Hbase. In terms of what use case this enables, I'd call out: Operational Data Hub/Real-Time BI, Customer 360 and several IoT related ones.</div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
Google Cloud Encryption</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivXY7H6o68udNmsGnkAay7rLfvQbkie8ZfumnM6Xh2vgqazMC6B-M8Yz6Mc0XILjWpdCHpOMraFMeCTQYkG3M0sKwWVBshxvHlt05oLxcERJhqhiQUuPLXFTYo52Q1ZPUAwe4RD70MKOU/s1600/google_cloud_security.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="1138" data-original-width="1106" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEivXY7H6o68udNmsGnkAay7rLfvQbkie8ZfumnM6Xh2vgqazMC6B-M8Yz6Mc0XILjWpdCHpOMraFMeCTQYkG3M0sKwWVBshxvHlt05oLxcERJhqhiQUuPLXFTYo52Q1ZPUAwe4RD70MKOU/s320/google_cloud_security.png" width="311" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Google CMEK allows data owners to have a multilayered security model that secures data and controls access to the data encryption keys. With this new capability, Pentaho users can use these custom encryption keys to access data in Google Cloud Storage and Google Big Query enhancing the security of the data. And we're very happy to say that we were able to test that it just works with no product change required! Damn, feels good when it happens 😁</div>
<div>
<br /></div>
<h3>
Edge to Cloud Processing</h3>
<div>
<br /></div>
<h4>
Adaptive Execution Layer (AEL) Improvements</h4>
<div>
<br /></div>
<div>
AEL is our cluster-version of the "Write Once, Run Everywhere", an abstraction layer where we currently have engines for Kettle (the classic one) and Spark.</div>
<div>
<br /></div>
<div>
Initially available in 7.1, we've been expanding it not only in terms of features but also in terms of vendor support. Here's the current matrix:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnfJwkqmqs_BrCRmru3Tb3M8rBE794yasU6rHVC72Y3k5J6f5axKIJPIQVkLmC6xGcheHOOOtqYVCFXFTmo1hmaeu0Lv4kA-E7eKh51pcWiOrZFml1zM4MMKAc9EsrnXg1tNGu6KsSg0A/s1600/ael_compat_matrix.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="388" data-original-width="1600" height="77" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhnfJwkqmqs_BrCRmru3Tb3M8rBE794yasU6rHVC72Y3k5J6f5axKIJPIQVkLmC6xGcheHOOOtqYVCFXFTmo1hmaeu0Lv4kA-E7eKh51pcWiOrZFml1zM4MMKAc9EsrnXg1tNGu6KsSg0A/s320/ael_compat_matrix.png" width="320" /></a></div>
<div>
<br /></div>
<div>
The current spark version supported is 2.3. There are many other point improvements to AEL. I won't go into details on them but here's a small overview:</div>
<div>
<br /></div>
<div>
<ul>
<li>Support for execution of MDI driven transformation via “ETL Metadata Injection” step</li>
<li>Support for sub-transformation steps Simple Mapping/Mapping (Transformation Executor was already supported)</li>
<li>Native Spark implementation for HBase and Hive</li>
<li>Support for S3 Cloud storage from AEL with native integration</li>
</ul>
</div>
<div>
<br /></div>
<div>
One relevant change though is that starting from 8.2 AEL is available only on the EE version. It's something that from the beginning was being debated, on opening it completely or not (the code was never available as we were actively changing the APIs and couldn't guarantee stability in external contributions). After looking at all the data we made the tough call to pull it to EE land :(</div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
Streaming AMQP</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfUbtWMeJWqXxycRKgtfw4-cIz_y5PM8ebP7CVrWZspOGdqli_BXnUEYjIaNxsQlr9lm1RVBDum6Rgxu1T5ZPgaup29zhP7xG7ovOWuLiUU1CWDowk8zzcf6-x-QBbT4-WejPKhj7Pqzs/s1600/streaming_pdi.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="772" data-original-width="1600" height="154" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhfUbtWMeJWqXxycRKgtfw4-cIz_y5PM8ebP7CVrWZspOGdqli_BXnUEYjIaNxsQlr9lm1RVBDum6Rgxu1T5ZPgaup29zhP7xG7ovOWuLiUU1CWDowk8zzcf6-x-QBbT4-WejPKhj7Pqzs/s320/streaming_pdi.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
8.0 introduced a new paradigm for handing streaming transformations in a continuous way. A new set fo steps work in conjunction with a newly introduced step (“Get Records from Stream”) to process micro-batch of continuous stream of records.</div>
<div>
<br /></div>
<div>
<div>
In the meantime, a few steps were introduced to ingest/produce streaming data from/to Kafka/MQTT/JMS and 8.1 introduced a mechanism to pass all streaming data together to downstream.</div>
</div>
<div>
<br /></div>
<div>
And now two steps were added: AMQP Consumer and AMQP Producer. Nuff said</div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Data Operations</h3>
<div>
<br /></div>
<h4>
Lineage improvements</h4>
<div>
<br /></div>
<div>
Getting to the point:</div>
<div>
<br /></div>
<div>
<ul>
<li>What’s New</li>
<ul>
<li>Architecture improvements for 3rd party lineage bridges (like IGC)</li>
<li>Add step and job entry "description" fields to lineage data output</li>
<li>Continued upgrading to Custom Lineage Analyzers for the following steps and job entries: Hadoop File Input & Output, Spark Submit, Mapping (sub-transformation), ETL Metadata Injection step (added relation to the sub-transformation being executed)</li>
</ul>
<li>Benefits</li>
<ul>
<li>Better and easier integration of 3rd party lineage bridges also for future partnering</li>
<li>Improve of using lineage information for documentation and compliance use cases</li>
<li>Expand completeness of data lineage steps and job entries</li>
</ul>
</ul>
</div>
<div>
<br /></div>
<h4>
Monitoring Status Page Update</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiGNulhiOjFoqj5myl7UPHqFDVUG1kknZRXmgLi5CF3vS89I3K6NOKtNfeOgkuELjKLUAwzZKpgPAGtgN1RWSlAZQ9TS0rGQs4y0ALDeovWHMC4gSfvLHCifMZlW1CV6HmBXTUWMLNBPA/s1600/monitoring_page.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="431" data-original-width="631" height="218" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiiGNulhiOjFoqj5myl7UPHqFDVUG1kknZRXmgLi5CF3vS89I3K6NOKtNfeOgkuELjKLUAwzZKpgPAGtgN1RWSlAZQ9TS0rGQs4y0ALDeovWHMC4gSfvLHCifMZlW1CV6HmBXTUWMLNBPA/s320/monitoring_page.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Probably one of the most asked feature of the decade. Our "vintage-look" pdi status page has been refreshed, and along with it some extra functionality </div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
OpenJDK support</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg694LDRqyZmFMC0K2FOXRtpJn4XLSpyIiEuFN5vKbDlV6KiDvaRsFLS7klTwoCQE_MdICNUi12Gv4I0AMpejONbd-GgdmZWCPem84rn1Wns6U9imnbqFE8xqSLM5u05TRyeLKoHOY6d14/s1600/openjdk.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="391" data-original-width="1600" height="78" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg694LDRqyZmFMC0K2FOXRtpJn4XLSpyIiEuFN5vKbDlV6KiDvaRsFLS7klTwoCQE_MdICNUi12Gv4I0AMpejONbd-GgdmZWCPem84rn1Wns6U9imnbqFE8xqSLM5u05TRyeLKoHOY6d14/s320/openjdk.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
If you've been waiting for this one as I have, you'll surely scream "FINALLY!!"! We now support of OpenJDK 8 JRE in server, AEL and client tools. </div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Advanced Analytics and Visualizations</h3>
<div>
There must be a reason why these 2 are connected on the same part - I just don't know why 😅</div>
<div>
<br /></div>
<h4>
Python Executor</h4>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieHBnQyBTCAGF-l0IoBi27U7RzkbXeVRwSocRd5fsvm6p-zq0oegzSqgDszBThcxKzEBTKWMTHtg-68iBd_dyL4HC8hf3fD4wtuQvoG2qlvPyug2ZT_m87N7yVS9wMaIkhruEMZFDacLA/s1600/python.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="990" data-original-width="1106" height="286" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieHBnQyBTCAGF-l0IoBi27U7RzkbXeVRwSocRd5fsvm6p-zq0oegzSqgDszBThcxKzEBTKWMTHtg-68iBd_dyL4HC8hf3fD4wtuQvoG2qlvPyug2ZT_m87N7yVS9wMaIkhruEMZFDacLA/s320/python.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
If you're an EE customer you'll be able to benefit from this refreshed, AEL compliant Python executor. Feature-wise, I'd call out:</div>
<div>
<ul>
<li>Automated ability to Get fields from Python script</li>
<li>Allows for multiple inputs (Row by Row or All Rows)</li>
<li>Ability to Pick a Python environment from one or more installed Python installations, i.e. virtual environments</li>
<li>Each Step gets its own Python session </li>
</ul>
</div>
<div>
<br /></div>
<div>
Used in conjunction with the existing R step and Spark Submit Job step to add overall Data Science offering and capabilities. </div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
PDI Data Science Notebook (Jupyter) Integration</h4>
<div>
Not a feature <i>per se</i> but an extremely useful consequence of the python step and all the improvements on data services.</div>
<div>
<br /></div>
<div>
Data scientists develop analytical models to achieve specific business results, and they perform much of their work within Notebooks, such as Jupyter. By using Pentaho Data Integration(PDI) and the new Python Executor step, Data Engineers can create data sets within PDI and make them available to be consumed within a Jupyter Notebook by the Data Scientist, as shown below in a collaborative workflow:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1KEO-8-ebpH24Zq9lP2mHLjugzpD9koM12jqtLfuPbBurNp_mwxhC-SetwXgK72cGrokxqHTVUoMtXonY5R4IeHw1RXOvE-w5STYcLb8MJe7PEy68bRJszVIMyFYFil0PEPCHfdAeWhk/s1600/jupyter1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="662" data-original-width="1456" height="145" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1KEO-8-ebpH24Zq9lP2mHLjugzpD9koM12jqtLfuPbBurNp_mwxhC-SetwXgK72cGrokxqHTVUoMtXonY5R4IeHw1RXOvE-w5STYcLb8MJe7PEy68bRJszVIMyFYFil0PEPCHfdAeWhk/s320/jupyter1.png" width="320" /></a></div>
<br />
<div>
<br /></div>
<div>
<br /></div>
<div>
Using PDI and the Python Executor step, the required data set is created using a PDI Data Service (virtual table), which can be consumed in the Jupyter Notebook, via a notebook template file . The file can be created programmatically via the Python Executor step in PDI, and pre-filled in with required connection info to the PDI Data Service.</div>
<div>
<br /></div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNXCD6YAj9T8-354Eox1tF4kU31yPn6CicbvFPJluunc5LkVUiqzmQckGvWtrPdsUSCcuVlP6se0HZ-mVkXJicm8HhbSqSpqIaOZHtigN6oLBM_NP3jriJ7bJyjze0zIODIaW3vPVTzQU/s1600/jupyter2.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="746" data-original-width="1198" height="199" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgNXCD6YAj9T8-354Eox1tF4kU31yPn6CicbvFPJluunc5LkVUiqzmQckGvWtrPdsUSCcuVlP6se0HZ-mVkXJicm8HhbSqSpqIaOZHtigN6oLBM_NP3jriJ7bJyjze0zIODIaW3vPVTzQU/s320/jupyter2.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<div>
Some technical considerations for the integration solution are as follows:</div>
<div>
<br /></div>
<div>
<ul>
<li>Pentaho Server needs to be running to host a PDI Data Service</li>
<li>PDI Spoon needs to be connected to the repository to save/deploy/edit the Data Service</li>
<li>PDI Data Service Client Jars need to be made available to be used by Jupyter Notebook</li>
<li>Compatible with Python 2.7.x or Python 3.5.x</li>
<li>Compatible with Jupyter Notebook 5.6.x</li>
<li>Python JDBC package dependencies include JayDeBeApi and jpype</li>
</ul>
</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
Streaming Vizualizations and CTools (Push)</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHsa_noddA_GXu_ihxHOlU0-32hvmMJNLptAu22I6ZXCOeM4qhyphenhyphen6Hhr5PvZKgq4AUKANXgAO3Qwxc_ljqCiWUi3fUaW0oOgW0cH7g3bUzPuN4LC4FBzHbDJC7FbLI2h-ok4ZT_du-O8as/s1600/realtime.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="293" data-original-width="543" height="172" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjHsa_noddA_GXu_ihxHOlU0-32hvmMJNLptAu22I6ZXCOeM4qhyphenhyphen6Hhr5PvZKgq4AUKANXgAO3Qwxc_ljqCiWUi3fUaW0oOgW0cH7g3bUzPuN4LC4FBzHbDJC7FbLI2h-ok4ZT_du-O8as/s320/realtime.png" width="320" /></a></div>
<div>
<br /></div>
<div>
I obviously love this one! We finished the connection between the (really awesome) Streaming Data Services all the way through CTools dashboards. Now the dashboard components that are ready for it (tables and charts are) will receive data as soon as it's ready, versus polling every N seconds.</div>
<div>
<br /></div>
<div>
<div>
When using CDE to create push-based streaming dashboards, the ‘Refresh period’ property for components and streaming over data services queries can now be left blank, as data can be continuously received and rendered as soon as new windows are available (see image below).</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNTAHuB86K1W5lhdhEm6ZPC1uodSCMklf0Sl4YWLmYbpcmwPFUcJuuNUbpfKeQTb98SBJeWeDWFEDyry4kMGreQN6mS2LJC5BGaIke92ZSfp0qOeOTB1ii9t0ZUfb-dZziU366Wp59wXg/s1600/cde.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="454" data-original-width="964" height="150" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhNTAHuB86K1W5lhdhEm6ZPC1uodSCMklf0Sl4YWLmYbpcmwPFUcJuuNUbpfKeQTb98SBJeWeDWFEDyry4kMGreQN6mS2LJC5BGaIke92ZSfp0qOeOTB1ii9t0ZUfb-dZziU366Wp59wXg/s320/cde.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
More information can be accessed here:</div>
<div>
<a href="https://help.pentaho.com/Documentation/8.2/Products/CTools/Create_Streaming_Service_Dashboard">https://help.pentaho.com/Documentation/8.2/Products/CTools/Create_Streaming_Service_Dashboard</a></div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Platform Updates</h3>
<div>
<br /></div>
<h4>
JSON Input updates</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj89QOCKSgdFwqzRObS8HfSIeF90SAMkL-F_RWr_CMxapH0B777ZStEPMl8AhmMhNgz6ccrkI5Z_BgYiqeOKy64skIEgRHbN48XZ1a_pQoy0GOmNuH4BhLqjPMANjIjeFoNdpwOu2Iwcfs/s1600/json.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="786" data-original-width="1214" height="207" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj89QOCKSgdFwqzRObS8HfSIeF90SAMkL-F_RWr_CMxapH0B777ZStEPMl8AhmMhNgz6ccrkI5Z_BgYiqeOKy64skIEgRHbN48XZ1a_pQoy0GOmNuH4BhLqjPMANjIjeFoNdpwOu2Iwcfs/s320/json.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
The JSON Input step now features a new Select Fields window for specifying what fields you want to extract from your source file. The window displays the structure of the source JSON file. Each field in the structure is displayed with a checkbox for you to indicate if it should be extracted from the file. You can also search within the structure for a specific field. Overall, these enhancements provide a drastic improvement in step usability.</div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
BA Chinese Language Localization for PUC</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSPYR9gKt1eXX2Z7PwGBDDJpzkMxL0Of4vBkracb_P2KnRPOXdLXQ-6rxr7wYaJeRKZr1AUERQf92lO70yKS2_5RePIlv5qRdKCz7ahBpe2VuPiEFSo31ZLYWjFtp9JD3_6zZ_p1UoFlE/s1600/chinese.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="558" data-original-width="688" height="259" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiSPYR9gKt1eXX2Z7PwGBDDJpzkMxL0Of4vBkracb_P2KnRPOXdLXQ-6rxr7wYaJeRKZr1AUERQf92lO70yKS2_5RePIlv5qRdKCz7ahBpe2VuPiEFSo31ZLYWjFtp9JD3_6zZ_p1UoFlE/s320/chinese.png" width="320" /></a></div>
<div>
<br /></div>
<div>
请不要让我失望,谷歌翻译</div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
Expanded Metadada Injection support</h4>
<div>
<br /></div>
<div>
Metadata injection enables the passage of metadata to transformation templates at runtime in order to drastically increase productivity, reusability, and automation of transformation workflow. This supports use cases like the onboarding of data from many files and tables to data lakes. In addition to existing metadata injection enabled steps, as of 8.2 you now can inject metadata into any field in the following Pentaho Data Integration (PDI) steps:</div>
<div>
<br /></div>
<div>
<div>
<ul>
<li>Get System Data</li>
<li>Execute Row SQL Script</li>
<li>Execute SQL Script</li>
<li>User Defined Java Class</li>
<li>AMQP Consumer</li>
<li>AMQP Producer</li>
<li>JMS Consumer</li>
<li>JMS Producer</li>
<li>Add a Checksum</li>
<li>Set Field Value</li>
<li>Set Field Value to a Constant</li>
</ul>
</div>
</div>
<div>
<br /></div>
<h4>
BA Analyzer Numeric Level Comparison Filters</h4>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_UxaAPKjOUgdx7mOVsvI2wT-0w8OxvF5gL5CbgtT6gWNQoHvYtAQkVpFXzpf8Hlhh03-a7kv1bB9XAGa0lj219IAp0QAzQXB9aWsXsuViQQG6ralfgXzVXKrWxP7IIaZmnEH-Yuncris/s1600/filters_in_analyzer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="496" data-original-width="768" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh_UxaAPKjOUgdx7mOVsvI2wT-0w8OxvF5gL5CbgtT6gWNQoHvYtAQkVpFXzpf8Hlhh03-a7kv1bB9XAGa0lj219IAp0QAzQXB9aWsXsuViQQG6ralfgXzVXKrWxP7IIaZmnEH-Yuncris/s320/filters_in_analyzer.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<div>
In 8.2, users can filter Analyzer reports using numeric level comparison filters, which provide an added degree of flexibility and productivity to Business Analytics customers. Previously, level filters treated all levels as text-based/non-numeric, and as such required filter criteria based on either a picklist or string matching. </div>
<div>
<br /></div>
<div>
The new filters for numeric levels include greater than, less than, greater than or equals, less than or equals, and between. For instance, as seen in the insurance example below, a numeric level representing monthly auto premiums can be filtered according to a numeric range, keeping only records and measure amounts (of individual customers) where the premium level is between $150 and $400 per month.</div>
<div>
<br /></div>
<div>
Additional considerations:</div>
<div>
<ul>
<li>The numeric level comparison filters can be parametrized for use with Dashboard Designer</li>
<li>The filters can be applied via the report URL</li>
<li>If you are working with a high cardinality level, it may make sense to optimize performance by adjusting the mondrian.olap.maxConstraints property (ensure joins are handled by the underlying database) and/or rounding your data to manage cardinality</li>
</ul>
</div>
</div>
<div>
<br /></div>
<h3>
Additional Enhancements</h3>
<div>
<br /></div>
<div>
Here are some not-so-minor other improvements that were done on the release:</div>
<div>
<br /></div>
<div>
<h4>
PDI Step & Job Entry Improvements</h4>
<div>
<ul>
<li>User Defined Java Class step: Support of Java 1.8</li>
<ul>
<li>Allow PDI users to make use of newer Java language features (e.g. enhanced for loops, lambda expressions, varargs, etc.)</li>
</ul>
<li>Text File Output step: Added support of variables in the "Split every...rows" property</li>
<ul>
<li>Improve creating of flexible output file sizes controlled by variables.</li>
</ul>
<li>FTPS job entries: Support "Advanced server protection level"</li>
<ul>
<li>All FTPS steps have been enhanced by supporting “private protection level”, so the data is secured by integrity and confidentiality.</li>
</ul>
<li>Rest Client step: Allow to provide custom content type headers.</li>
<ul>
<li>Many REST servers require custom content types to be sent to them. In particular W3C Semantic compliant data stores such as Allegrograph and MarkLogic Server. </li>
</ul>
<li>Text File Input Step: Provide the full stack trace when a file cannot be opened</li>
<ul>
<li>The full stack trace will provide very valuable debugging information and allow root cause analysis of problems to resolved them more quickly.</li>
</ul>
<li>Calculator step: Added exceptions when a file is not found.</li>
<ul>
<li>Instead of providing bad data when a file is not available, the process ends with an error to notify the user of the issue.</li>
</ul>
</ul>
</div>
</div>
<h4>
</h4>
<h4>
BA Improvements</h4>
<div>
<div>
<br /></div>
<div>
<ul>
<li>PUC Upload/Download: Users with ‘publish content’ permission can now upload/download files to PUC</li>
<ul>
<li>No longer need to rely on a few users with complete ‘admin’ rights to move content btwn environments</li>
</ul>
<li>Scheduling Access: PUC users without scheduling permissions can no longer see the scheduling perspective</li>
<ul>
<li>More logical permissions and user experience for BA customers</li>
</ul>
<li>MDX Performance: MDX optimizations for some scenarios that incl. subtotals, numeric filters, and percentages</li>
<ul>
<li>Better performance in some Analyzer/Mondrian query scenarios</li>
</ul>
<li>Analyzer Business Groups: Global setting option to expand or collapse Analyzer business groups</li>
<ul>
<li>Long lists of fields can be rolled up by default when a report is opened, reducing scrolling / improving UX</li>
</ul>
<li>Analyzer Numeric Dimension Filters: (*Stretch Goal*) Comparison filters ( < , > , btwn, …) on numeric levels (i.e. age, credit score, customer id)</li>
<ul>
<li>Much greater flexibility to query data with numeric levels (i.e. show me sales for customers between ages of 18 and 30). Previously every distinct level value would have to be manually added to an include filter criteria.</li>
</ul>
</ul>
</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div>
<a href="https://community.hitachivantara.com/docs/DOC-1009931-downloads">Get it here</a> and Enjoy!!!</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com5tag:blogger.com,1999:blog-6644329693530300467.post-74422680149445919822018-07-27T13:15:00.004+01:002018-07-27T13:15:56.489+01:00Pentaho Community Meeting - PCM18! Bologna, Italy, November 23-25!<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://it-novum.com/fileadmin/user_upload/Hintergrundbilder/headerbanner-PCM18.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="267" data-original-width="800" height="212" src="https://it-novum.com/fileadmin/user_upload/Hintergrundbilder/headerbanner-PCM18.jpg" width="640" /></a></div>
<br />
<br />
PCM 18!!<br />
<br />
If you've been in one, no more words are needed, <a href="https://it-novum.com/en/pcm18/?utm_source=pedroalves-bi.blogspot.com&utm_medium=social&utm_campaign=pcm18?wm=pcm18-pedroalvesblog">just go ahead and register</a>! If you don't know what I'm talking about,<b> <a href="https://it-novum.com/en/pcm18/?utm_source=pedroalves-bi.blogspot.com&utm_medium=social&utm_campaign=pcm18?wm=pcm18-pedroalvesblog">just go ahead and register</a></b> as well!<br />
<br />
It's the best example of what Pentaho - how part of Hitachi Vantara - is all about. A very passionate group of people that are absolutely world class at what they do and still know how to spend a good time!<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://it-novum.com/fileadmin/_processed_/csm_Gruppenbild-Pentaho-Community-Meeting-2017_2d07006266.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" data-original-height="292" data-original-width="800" height="145" src="https://it-novum.com/fileadmin/_processed_/csm_Gruppenbild-Pentaho-Community-Meeting-2017_2d07006266.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>PCM17 group photo</i></td></tr>
</tbody></table>
<br />
<br />
Now shamelessly copy-pasting the content from <a href="https://it-novum.com/en/pcm18/?utm_source=pedroalves-bi.blogspot.com&utm_medium=social&utm_campaign=pcm18?wm=pcm18-pedroalvesblog">it-novum</a>:<br />
<br />
<h2>
Pentaho Community Meeting 2018</h2>
<br />
<br />
Pentaho Community Meeting 2018 will take place in Bologna from November 23-25. It will be organized by <a href="https://community.hitachivantara.com/groups/pug-italia">Italia Pentaho User Group</a> and by <a href="https://it-novum.com/en/pcm18/?utm_source=pedroalves-bi.blogspot.com&utm_medium=social&utm_campaign=pcm18?wm=pcm18-pedroalvesblog">it-novum</a>, the host of PCM17. As always, it will be a 3-days event full of presentations, networking and fun and we invite Pentaho users of every kind to participate!<br />
<br />
For PCM18 we will meet in the beautiful city of Bologna. The guys of Italia User Group will take care of the venue and the program. With Virgilio Pierini as group representative we not only have a Pentaho enthusiast but also a native of Bologna guiding us to the beautiful corners of the hometown of Europe’s oldest university!<br />
<br />
<h3>
What is Pentaho Community Meeting?</h3>
<br />
Pentaho Community Meeting is an informal gathering for Pentaho users from around the world. We meet to discuss the latest and greatest in Pentaho products and exciting geek stuff (techie track) as well as best practices of Pentaho implementations and successful projects (business track). Read this summary of Pentaho Community Meeting 2017 to learn more.<br />
<br />
PCM18 is open to everyone who does something with Pentaho (development, extensions, implementation) or plans to do data integration, analytics or big data with Pentaho. Several Pentaho folks – architects, designers, product managers – will share their latest developments with us.<br />
<br />
The event is community-oriented and open-minded. There’s room for networking and exchanging ideas and experiences. Participants are free to break off into groups and work together.<br />
<br />
<h3>
Call for Papers</h3>
<br />
For sure, this is intended to be a community event - for the community and by the community. To register your proposal for the agenda, please use the contact form to send a brief description including your name and title in English until September 30th.<br />
<br />
<h3>
Agenda</h3>
<br />
The agenda will be updated continuously, so stay tuned for updates! All updates will be posted on twitter, too.<br />
<br />
<h4>
Friday, November 23 | Hackathon</h4>
<br />
We start the three-day PCM with a hackathon, snacks and drinks. After a 2-hour hackathon, a highly esteemed jury will award the most intelligent/awkward/funny hacks.<br />
<br />
<h4>
Saturday, November 24 | Conference Day</h4>
<div>
<br /></div>
<div>
Still a lot to be determined! We're still receiving papers</div>
<br />
<ul>
<li>Welcome speech | Stefan Müller and the org team</li>
<li>The future of Pentaho in Hitachi Vantara | Pedro Alves, Hitachi Vantara</li>
<li>What's new in PDI 9.0 | Jens Bleuel, Hitachi Vantara</li>
<li>Useful Kettle plugins | Matt Casters, Neo4j (and founder of Kettle)</li>
<li>IoT and AI: Why innovation is a societal imperative | Wael Elrifai, VP for Solution Engineering - Big Data, IOT & AI, Hitachi Vantara</li>
<li>Pentaho at CERN | Gabriele Thiede, CERN</li>
<li>Pentaho User Group Italia</li>
<li>SSBI (Self Service BI ) - Pentaho Plugin Update | Pranav Lakhani, SPEC INDIA</li>
<li>Scaling Pentaho Server with Kubernetes | Diethard Steiner</li>
<li>Capitalizing on Lambda & Kappa Architectures for IoT with Pentaho | Issam Hizaji, Lead Sales Engineer, Data Analytics & IoT | Emerging & Southern</li>
</ul>
<br />
After the lunch, everybody splits up to join the business or the techie track.<br />
<br />
<h4>
Sunday, November 25 | Social Event</h4>
Brunch, sightseeing and... let´s see!<br />
<br />
<br />
----<br />
<br />
Anyway, believe me, you want to go! <a href="https://it-novum.com/en/pcm18/?utm_source=pedroalves-bi.blogspot.com&utm_medium=social&utm_campaign=pcm18?wm=pcm18-pedroalvesblog">GO REGISTER HERE</a>!<br />
<br />
<br />Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com1tag:blogger.com,1999:blog-6644329693530300467.post-6490144144594638982018-05-16T17:15:00.000+01:002018-05-16T17:15:44.627+01:00Pentaho 8.1 is available<br />
<h1 style="line-height: 115%; margin-bottom: 6.0pt; mso-pagination: lines-together;">
Pentaho
8.1<b> is available<o:p></o:p></b></h1>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "times new roman" , serif; font-size: 12.0pt;">The
team has once again over delivered on a dot release! Below are what I think are
the many highlights of Pentaho 8.1 as well as a long list of additional
updates. <o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-family: "times new roman" , serif; font-size: 12.0pt;">If
you don’t have time to read to the end of my very long blog, just save some
time and download it now. </span><span style="font-family: inherit;"><a href="https://www.hitachivantara.com/en-us/products/big-data-integration-analytics/pentaho-trial-download.html?source=pedroalves-bi" style="background-color: white; display: inline; outline: none; text-align: justify; text-decoration-line: none; transition: color 0.3s;">Go get your Enterprise Edition or trial version from the usual places</a></span><br />
<span style="font-family: inherit;"><br style="background-color: white; text-align: justify;" /><span style="background-color: white; text-align: justify;">For CE, you can find it </span><a href="https://community.hds.com/docs/DOC-1009931-downloads" style="background-color: white; display: inline; outline: none; text-align: justify; text-decoration-line: none; transition: color 0.3s;">on the community home</a><span style="background-color: white; text-align: justify;">!</span></span></div>
<div class="MsoNormal">
<br /></div>
<h2 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Cloud<o:p></o:p></b></h2>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
One of the biggest themes of the release: Increased support
for Cloud. A lot of vendors are fighting for becoming the best providers, and
what we do is try to make sure Pentaho users watch all that comfortably sitting
on their chairs, having a glass of wine, and really not caring about the
outcome. Like in a lot of areas, we want to be agnostic – which is not saying
that we’ll leverage the best of each – and really focus on logic and execution.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
It’s hard to do this as a one time effort, so we’ve been
adding support as needed (and by “<i>as needed” </i>I really mean based on the
prioritization given by the market and our customers). A big focus of this
release was Google and AWS: <o:p></o:p></div>
<div class="MsoNormal">
<span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shapetype
id="_x0000_t75" coordsize="21600,21600" o:spt="75" o:preferrelative="t"
path="m@4@5l@4@11@9@11@9@5xe" filled="f" stroked="f">
<v:stroke joinstyle="miter"/>
<v:formulas>
<v:f eqn="if lineDrawn pixelLineWidth 0"/>
<v:f eqn="sum @0 1 0"/>
<v:f eqn="sum 0 0 @1"/>
<v:f eqn="prod @2 1 2"/>
<v:f eqn="prod @3 21600 pixelWidth"/>
<v:f eqn="prod @3 21600 pixelHeight"/>
<v:f eqn="sum @0 0 1"/>
<v:f eqn="prod @6 1 2"/>
<v:f eqn="prod @7 21600 pixelWidth"/>
<v:f eqn="sum @8 21600 0"/>
<v:f eqn="prod @7 21600 pixelHeight"/>
<v:f eqn="sum @10 21600 0"/>
</v:formulas>
<v:path o:extrusionok="f" gradientshapeok="t" o:connecttype="rect"/>
<o:lock v:ext="edit" aspectratio="t"/>
</v:shapetype><v:shape id="_x0000_i1046" type="#_x0000_t75" style='width:453.6pt;
height:162pt;visibility:visible'>
<v:imagedata src="file:///C:/Users/palves/AppData/Local/Temp/msohtmlclip1/01/clip_image001.jpg"
o:href="file:///C:/Users/palves/Downloads/Image_2"/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="text-align: center;">
<span id="docs-internal-guid-4dcd0e5c-6947-9385-92fa-77bce85aa7ed"><span style="font-family: "calibri"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><img height="215" src="https://lh4.googleusercontent.com/v4JHvCLRXAtdN1eB7xsg_l-7xEd7PF-uIJfScgNfZKrHTrb6w8DNesQ97u6SaK9u3M4Z_LTv7hvZjei6hhhq14qnNl-gUYz3QrAxPXo1sqk9vWGSJNVvIoUxMP053mT1s88eHRQUTzBdm6RWvQ" style="border: none; transform: rotate(0rad);" width="602" /></span></span></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Google
Storage (EE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Google Cloud Storage is a RESTful unified storage for
storing and accessing data on Google's infrastructure. PDI support for import
and export Data To/From Cloud Storage is now done through a new VFS driver (<i>gs://</i>).
You may even use it on the several steps that support it as well as browse it’s
contents.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
These are the roles required on Google Storage for this to
work:<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Storage Admin<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Storage Object Admin<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Storage Object Creator<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Storage Object Viewer<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In terms of authentication, you’ll need the following
environment variable defined:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-bottom: 8.0pt; margin-left: .6in; margin-right: .6in; margin-top: 10.0pt; text-indent: -.6in;">
<i><span style="color: #404040;">GOOGLE_APPLICATION_CREDENTIALS="/opt/Pentaho81BigQuery.json“<o:p></o:p></span></i></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
From this point on, just treat it as a normal VFS source.<o:p></o:p></div>
<span id="docs-internal-guid-c24dddac-6947-c914-a650-dadafaaf7485"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-c24dddac-6947-c914-a650-dadafaaf7485"><img height="269" src="https://lh4.googleusercontent.com/ytg8bFaw8hgI2Jh8pJ3fqqVt_0hNOmH1qmNQRZhhwClmm9raZZYQjoo0-nDA8xLdi1jaBMIQCxDOJl6V24LmXFV8LxglnBz4tzfsB5SwRn6BoBHTfcCKhdTB8qtT2ctw3cQ7XqxqYogUcXQ9Ig" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="305" /></span></div>
<span id="docs-internal-guid-c24dddac-6947-c914-a650-dadafaaf7485">
</span>
<br />
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<span style="mso-spacerun: yes;"> </span>Google BigQuery – JDBC Support<span style="mso-spacerun: yes;"> </span>(EE/CE)<b><span style="font-family: "times new roman" , serif;"><o:p></o:p></span></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">BigQuery is Google's
serverless, highly scalable, low cost enterprise data warehouse. Fancy name for
a database, and that’s how we treat it.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">In order to connect to it
first we need the appropriate drivers. Steps here are pretty simple:<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l1 level1 lfo2; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="mso-fareast-font-family: Calibri;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Download free driver:<span style="mso-spacerun: yes;"> </span><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">https</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">://</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">cloud</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">.</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">google</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">.</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">com</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">/</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">bigquery</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">/</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">partners</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">/</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">simba</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">-</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">drivers</span></a><a href="https://cloud.google.com/bigquery/partners/simba-drivers/"><span style="color: #0563c1;">/</span></a> <o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l1 level1 lfo2; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "times new roman" , serif; font-size: 10.0pt;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->Copy
google*.* files from Simba driver to
/pentaho/design-tools/data-integration/libs folder<span style="font-family: "times new roman" , serif; font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal" style="margin-left: .5in; text-indent: -.5in;">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">Host Name will default to </span><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">https</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">://</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">www</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">.</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">googleapis</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">.</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">com</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">/</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">bigquery</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">/</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">v</span></a><a href="https://www.googleapis.com/bigquery/v2"><span style="color: #0563c1; font-size: 10.0pt;">2</span></a><span style="font-size: 10.0pt;"> but your mileage may
vary.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">Unlike the previous item,
authentication doesn’t use the previously defined environment variable as
does Google VFS. Authentication here is done at the JDBC driver level, though a
driver option, <i>OAuthPvtKeyPath</i>, set in the
Database Connection Option and the you need to point to
the Google Storage certificate through the P12 key format.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">The following Google BigQuery
roles are required:<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l2 level1 lfo3; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="mso-fareast-font-family: Calibri;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->BigQuery Data Viewer<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l2 level1 lfo3; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="mso-fareast-font-family: Calibri;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->BigQuery User<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<span id="docs-internal-guid-ef087b7d-6948-03f8-4a94-f49c1e9ffd48"><span style="font-family: "calibri"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><img height="228" src="https://lh3.googleusercontent.com/XHUpgBEWXUFHK4W01dG-cbsBSIqYW5gRtLtMA5D1ZfuDLzcdkI1DmDAvCRv7CYZCHxpTNts4IrzS_QBpcrvN0BaOZp37VqVoSzAoIiAXq2pmRvzIEw6rHMErkpK5FCgE73BPExnNe4XMCF8t4Q" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="303" /></span></span></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
Google
BigQuery – Bulk Loader<span style="mso-spacerun: yes;"> </span>(EE)<b><span style="font-family: "times new roman" , serif;"><o:p></o:p></span></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">While you can use a regular
table output to insert data into BigQuery that’s going to be slow as hell (who
said hell was slow? This expression makes no sense at all!). So we’ve added a
step for that: Google BigQuery Loader.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">This step leverages google’s
loading abilities, and is processed out on Google, not on PDI. So the data,
that has to be either in Avro, JSON or CSV has to be previously copied to
Google Storage. From that point on is pretty straightforward. Authentication is
done via the </span><i>GOOGLE_APPLICATION_CREDENTIALS</i> environment variable
point to the Google JSON file.<span style="font-size: 10.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<span id="docs-internal-guid-fa43dace-6948-38e5-72e6-e1d8d5b77f96"><span style="font-family: "calibri"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><img height="416" src="https://lh3.googleusercontent.com/HIQq7aqH74gyYSrQxXdjJzHBDBpF53juwS8boEBpxpKqBzwLPo_BCaYMo_6dy6OdrtfvLS3LzRq8tbxJOHCtptP1OFeE-tKtOdal-Hm6_JLB0nB1rVcMqOW6XCca6pDegyPjSZ2WNQ_zWR-Kqg" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="303" /></span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-top: 2.0pt; mso-pagination: widow-orphan lines-together; page-break-after: avoid;">
<span style="color: #1f3863; font-size: 12.0pt;">Google
Drive<span style="mso-spacerun: yes;"> </span>(EE/CE)</span><span style="color: #1f3863; font-family: "times new roman" , serif; font-size: 12.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">While Google Storage will
probably be seen more frequently in production scenarios, we also added support
for Goggle Drive, a file storage and synchronization service, allows users to
store files on their servers, synchronize files across devices, and share
files.<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span style="font-size: 10.0pt;">This is also done through a
VFS driver, but given it’s a per user authentication a few steps need to be
fulfilled to leverage this support:<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Copy your
Google client_secret.json file into (The Google
Drive option will not appear as a Location until you copy
the client_secret.json file into the credentials directory
and restart)<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.5in; mso-list: l0 level2 lfo1; tab-stops: 1.25in 1.5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "courier new"; mso-fareast-font-family: "Courier New";"><span style="mso-list: Ignore;">o<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->Spoon: data-integration/plugins/pentaho-googledrive-vfs/credentials directory,
and restart spoon. <o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.5in; mso-list: l0 level2 lfo1; tab-stops: 1.25in 1.5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "courier new"; mso-fareast-font-family: "Courier New";"><span style="mso-list: Ignore;">o<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->Pentaho
Server:
pentaho-server/pentaho-solutions/system/kettle/plugins/pentaho-googledrive-vfs/credentials directory
and restart the server<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Select Google Drive as
your Location. You are prompted to login to your Google account.<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Once you have logged in, the Google Drive
permission screen displays.<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Click Allow to access your Google
Drive Resources.<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->A new file called <i>StoredCredential</i> will
be added to the same place where you had the <i>client_secret</i>.<i>json</i>
file. This file will need to be added to the Pentaho Server credential location
and that authentication will be used<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Analytics
over BigQuery<span style="mso-spacerun: yes;"> </span>(EE/CE, depending on the
tool used)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This JDBC connectivity to Google BigQuery, as defined
previously for Spoon, can also be used throughout all the other Business
Analytics browser and client tools – Analyzer, CTools, PIR, PRD, modeling
tools, etc. Some care has to be taken here, though, as BigQuery’s pricing is
related to 2 factors:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Data stored<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l0 level1 lfo1; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Data queried<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
While the first one is relatively straightforward, the
second one is harder to control, as you’re charged according to total data
processed in columns selected. For instance, a ‘<i>select *</i>’ query should
be avoided if only specific columns are needed. To be absolutely clear, this
has nothing to do with Pentaho, these are Google BigQuery pricing rules.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
So ultimately, and a bit like we need to do on all databases
/ data warehouses, we need to be smart and work around the constraints (usually
speed and volume, on this case price as well) to leverage best what these
technologies have to offer. Some examples are given here:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->By default, there is BigQuery caching and cached
queries are free. For instance, if you run a report in Analyzer, clear the
Mondrian cache, and then reload the report, you will not be charged (thanks to
the BigQuery caching)<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Analyzer: Turn off auto refresh, i.e, this way
you design your report layout first, including calculations and filtering,
without querying the database automatically after each change<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Analyzer: Drag in filters before levels to
reduce data queried (i.e. filter on state = California BEFORE dragging city,
year, sales, etc. onto canvas)<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Pre-aggregate data in BigQuery tables so they
are smaller in size where possible (to avoid queries across all raw data)<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; font-size: 12.0pt;"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->GBQ
administrators can set query volume limits by user, project, etc. (quotas)<span style="font-family: "times new roman" , serif; font-size: 12.0pt;"><o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>AWS
S3 Security Improvements (IAM) (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
PDI is now able to get IAM security keys from the following
places (in this order):<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l4 level1 lfo5; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="mso-fareast-font-family: Calibri;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Environment Variables<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l4 level1 lfo5; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="mso-fareast-font-family: Calibri;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Machine’s home directory<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l4 level1 lfo5; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="mso-fareast-font-family: Calibri;"><span style="mso-list: Ignore;">3.<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->EC2 instance profile<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This added flexibility helps accommodate different AWS
security scenarios, such as integration with S3 data via federated SSO from a
local workstation, by providing secure PDI read/write access to S3 without
making user provide hardcoded credentials.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
The IAM user secret key and access key can be stored in one
place so they can be leveraged by PDI without repeated hardcoding in Spoon.
These are the environment variables that point to them:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-bidi-font-style: italic; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><i>AWS_ACCESS_KEY_ID<o:p></o:p></i></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-bidi-font-style: italic; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]--><i>AWS_SECRET_ACCESS_KEY<o:p></o:p></i></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-eb60a434-6948-74fc-2d09-5c98fc19b692"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-eb60a434-6948-74fc-2d09-5c98fc19b692"><img height="285" src="https://lh5.googleusercontent.com/9A-sIhlwcA4-ib3u3xrLPm3oBCR19byRCW1l1t09AkGtpeal4TEwKl5DjQswpi7jml-v8UnG8JVwpwfbadQbETO08dYZDousIsG8OTQu08Eqb8yAaBB-uF3G5UCvQpZMzkAXagVsx71KyedtjQ" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="303" /></span></div>
<span id="docs-internal-guid-eb60a434-6948-74fc-2d09-5c98fc19b692">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<h2 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
Big
Data / Adaptive Execution Layer (AEL) Improvements <b><span style="font-family: "times new roman" , serif;"><o:p></o:p></span></b></h2>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-79b7086b-6948-a89b-3a4d-ef2f114de2d8"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-79b7086b-6948-a89b-3a4d-ef2f114de2d8"><img height="134" src="https://lh6.googleusercontent.com/xnoxWuyTNTJiKF1hBsI-4B4wPXzcs-3puv-ibobBiaWtZ076ykkZNVqd1ZAke_2IsyjljGjVgbnT2icMsMwZjUjgkFjvG3bPwSmXbzPBQTQHKJT_YPpb8Bzo-QVcQVCC0TieIN8V3nnSgbUv2A" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="303" /></span></div>
<span id="docs-internal-guid-79b7086b-6948-a89b-3a4d-ef2f114de2d8">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Bigger
and Better (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
AEL provides spectacular scale out capabilities (or is it
scale up? I can’t cope with these terminologies…) by seamlessly allowing a very
big transformation to leverage a clustered processing engine. <o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Currently we have support for Spark through the AEL layer,
and throughout the latest releases we’ve been improving it in 3 distinct areas:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Performance and resource optimizations<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l3 level2 lfo4; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "courier new"; mso-fareast-font-family: "Courier New";"><span style="mso-list: Ignore;">o<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->Added
Spark Context Reuse that, under certain circumstances can speed up startup
performance on the range to 5x faster, proving specially useful under
development conditions<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l3 level2 lfo4; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "courier new"; mso-fareast-font-family: "Courier New";"><span style="mso-list: Ignore;">o<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->Spark
History Server integration, providing a centralized administration, auditing
and performance reviews of the transformations executed in Spark<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: 1.0in; mso-list: l3 level2 lfo4; tab-stops: .75in 1.0in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "courier new"; mso-fareast-font-family: "Courier New";"><span style="mso-list: Ignore;">o<span style="font: 7.0pt "Times New Roman";"> </span></span></span><!--[endif]-->Ability
to passing down to the cluster customized spark properties, allowing a
finer-grained control of the execution process<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Increased support for native steps (eg,
leveraging the spark specific group by instead of the PDI engine one)<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Adding support for more cloud vendors – and we
just did that for EMR 5.9 and MapR 5.2<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This is the current support matrix for Cloud Vendors:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-76bbf566-6949-1300-79c8-94027978e4cb"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-76bbf566-6949-1300-79c8-94027978e4cb"><img height="78" src="https://lh4.googleusercontent.com/zQg-M9I8MAYaGicP3JIDHztiIkwA-0q_bkMwKOImZJ7sWIjyRT-NFsjZNqpb_n2-8Mtte370AT-L8Z09-oK6S4sRLsrJJTKVkBKnIgt6GBLBat70kTs-2eRHRy-xEioly9TgXI-QvXKvua_GgQ" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="303" /></span></div>
<span id="docs-internal-guid-76bbf566-6949-1300-79c8-94027978e4cb">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Sub
Transformation support (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This one is big, as it was the result of a big and important
refactor on the kettle engine. AEL Now supports executing sub transformations
through the <i>Transformation Executor</i> step, a long-standing request since
the times of good-old PMR (Pentaho Map Reduce)<o:p></o:p></div>
<div class="MsoNormal" style="text-align: center;">
<span id="docs-internal-guid-798df234-6949-80e4-b96f-60669de72bb8"><span style="font-family: "calibri"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><img height="281" src="https://lh4.googleusercontent.com/KlqSqp7TG6Ntr04vLGo2ddyN9WOkynzurHYMkZJIfvn3uJ0e4cusVGppt-UjvyqLqwjzbqckBM-cqmNL7VjG0Dh_qNLbK-s8vvwVNo0hqCIA0uwDJP2CUshLgDnddS9VpSFTro6fXX0MBGXA_w" style="border: none; transform: rotate(0rad);" width="305" /></span></span></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shape id="_x0000_i1039" type="#_x0000_t75"
style='width:230.4pt;height:212.4pt;visibility:visible'>
<v:imagedata src="file:///C:/Users/palves/AppData/Local/Temp/msohtmlclip1/01/clip_image015.jpg"
o:href="file:///C:/Users/palves/Downloads/Image_10"/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Big
Data formats: Added support for Orc (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Not directly related to AEL, but most of the use cases where
we want the AEL execution we’ll need to input data in a big data specific
format. In previous releases we added support for <i>Parquet</i> and <i>Avro</i>,
and we now added support for ORC (Optimized Record Columnar), a format favored
by Hortonworks.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Like the others, Orc will be handled natively when
transformations are executed in AEL<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<span style="mso-no-proof: yes;"><!--[if gte vml 1]><v:shape id="_x0000_i1038" type="#_x0000_t75"
style='width:228.6pt;height:246.6pt;visibility:visible'>
<v:imagedata src="file:///C:/Users/palves/AppData/Local/Temp/msohtmlclip1/01/clip_image017.jpg"
o:href="file:///C:/Users/palves/Downloads/Image_9"/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--></span><o:p></o:p></div>
<div class="MsoNormal" style="text-align: center;">
<o:p><img height="327" src="https://lh4.googleusercontent.com/bcleI_8E8hhWbAL2C_7KPanSDWL1fMqzqMkjoyz4y-1qPNM8nP04bK46CFMihWIxO9z7fDqF8w1opjP-F4cNEeViQlPzFNiBeVBjkmFcmZoI2HiNvFoEF-VQGpl4oP5NGMWjd5gmfcenGWSvZg" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="303" /> </o:p></div>
<h2 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Worker
Nodes (EE)<o:p></o:p></b></h2>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-6008c145-694a-09bc-d5a2-24f2f68de9b7"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-6008c145-694a-09bc-d5a2-24f2f68de9b7"><img height="80" src="https://lh5.googleusercontent.com/XrmRjZxzXs7zymbFWM0x32GMFfLwFo0QcKkVDrPpZ9c07lAet9BtL6a8osgzUdW5x3YYIMe4cYRjXuJkxkjZ2C2JHlZFWnW70F537305A-tDQUfYhBmz2EHZqTdv5eTSJk2uwB7lBj7ymAeGbQ" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="302" /></span></div>
<span id="docs-internal-guid-6008c145-694a-09bc-d5a2-24f2f68de9b7">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Jumping from scale-out to scale-up (or the opposite, like I
mentioned, I never know), we continue to do lots of improvements on the Worker
Nodes project. This is an extremely strategic project for us as we integrate
with the larger Hitachi Vantara portfolio.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Worker nodes allow you to execute Pentaho work items, such
as PDI jobs and transformations, with parallel processing and dynamic
scalability with load balancing in a clustered environment. It operates easily
and securely across an elastic architecture, which uses additional machine
resources as they are required for processing, operating on premise or in the
cloud.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
It uses the Hitachi Vantara Foundry project, that leverages
popular technologies under the hood such as Docker (Container Platform),
Chronos (Scheduler) and Mesos/Marathon (Container Orchestration).<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
For 8.1 there are several other improvements:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Improvements tn Monitoring, with accurate
propagation of Work Items status for monitoring<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Performance improvements by optimizing the
startup times for executing the work items<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Customizations are now externalized from docker
build process<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Job clean up functionality<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-cf195404-694a-2fc9-7fec-48fe0988c466"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-cf195404-694a-2fc9-7fec-48fe0988c466"><img height="125" src="https://lh6.googleusercontent.com/SHXzxvJEfLp5eH_Vb0PKhGvxImOsC7fUWd6snpA8Yn0IZfp3BSO_3iJBk3acUDpM35KJUW4FqNGoxVQ-N1mAn7dMuHeBczd97vUkN-QV3YHBehEza84lesyOmDM4I435Cm8oeSyoLZsLpe_TZA" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="302" /></span></div>
<span id="docs-internal-guid-cf195404-694a-2fc9-7fec-48fe0988c466">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<h2 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Streaming<o:p></o:p></b></h2>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-1f8d2bf0-694a-5311-5bd2-e9e9113d6e71"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-1f8d2bf0-694a-5311-5bd2-e9e9113d6e71"><img height="160" src="https://lh3.googleusercontent.com/QOWrV9qPn0jlEsCQIZ9TpMeL6bOtpVTQ_aQZ_1doxIeRVbvbpk6WJn76NOefW2ItM91azMniT0FwtuKPpnIJlSvs7l0R03JeA4IUAt4LDoqmTsFQw052P1jP_bHcpv4qm6oDQ0xbm0rv1S3_Ew" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="305" /></span></div>
<span id="docs-internal-guid-1f8d2bf0-694a-5311-5bd2-e9e9113d6e71">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In Pentaho 8.0 we introduced a new paradigm to handle
streaming datasources. The fact that it’s a permanently running transformation
required a different approach: The new streaming steps define the windowing
mode and point to a sub transformation that will then be executed on a micro
batch approach.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
That works not only for ETL within the kettle engine but
also in AEL, enabling spark transformations to feed from Kafka sources.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>New
Streaming Datasources: MQTT, and JMS (Active MQ / IBM MQ) (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Leveraging on the new streaming approach, there are 2 new
steps available – well, one new and one (two, actually) refreshed.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
The new one is MQTT – Message Queuing Telemetry Transport -
an ISO standard publish-subscribe-based messaging protocol that works on top of
the TCP/IP protocol. It is designed for connections with remote locations where
a "small code footprint" is required or the network bandwidth is
limited.<span style="mso-spacerun: yes;"> </span>Alternative IoT centric protocols
include AMQP, STOMP, XMPP, DDS, OPC UA, WAMP<o:p></o:p></div>
<span id="docs-internal-guid-530c65b4-694a-79af-404f-24f27bb39fd4"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-530c65b4-694a-79af-404f-24f27bb39fd4"><img alt="Image result" height="231" src="https://lh6.googleusercontent.com/CVTqeReaCAIjUHVsGQmog5EN6_DlqkiRKzs7K0wmUSGHzf-CStx_rC3-veGSHu1Bbd04SYOn2pCjGdeR5auoyRvqAhcZ7fpqIORKqOt_njNHJMYwQXmBk7DCzp8tpgVNTasMZW2pMVNR-wYdQA" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="303" /></span></div>
<span id="docs-internal-guid-530c65b4-694a-79af-404f-24f27bb39fd4">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
There are 2 new steps – MQTT Input and MQTT Output, that
connect with the broker for consuming and publishing back the results.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Other than this new, IoT centered streaming source, there
are 2 new steps, JMS Input and JMS Output. These steps replace the old JMS
Consumer/Producer and the IBM Websphere MQ steps, supporting, in the new mode
the following message queue platforms:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->ActiveMQ<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->IBM MQ<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-top: 2.0pt; mso-pagination: widow-orphan lines-together; page-break-after: avoid;">
<span style="color: #1f3863; font-size: 12.0pt;">Safe Stop
(EE/CE)<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This new paradigm to handle streaming sources introduced a
new challenge that we never had to face. Usually, when we triggered jobs and
transformations, they had a well defined start and end; Our stop functionality
was used when we wanted to basically kill a running process because something
was not going well.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
However, on these streaming use cases, a transformation may
never finish. So stopping a transformation the way we’ve always done – by
stopping all steps at the same time – could have unwanted results.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
So we implemented a different approach – We added a new
option to safe stop a transformation implemented within Spoon, Carte and the
Abort step, that instead of killing all the step threads, stops the input steps
and lets the other steps gracefully finish the processing, so no records
currently being processed are lost.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-80366f62-694a-9cd3-718d-d5bae03a49fa"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-80366f62-694a-9cd3-718d-d5bae03a49fa"><img height="96" src="https://lh5.googleusercontent.com/qN33M2vpkU_7Sy4WlGqtUFZNGSFT0X63tkkaF9MzuEBimTnq9C_rVbk1WvxD6jcRfnkV8IEdLq_vL4GC-TNpmjHzNhrN1I2Y8POv7ZH1EK5DYX3y8BsyJeGn0YDZskcGHlPrHU_DzVY-J38zUg" style="border: 0pt solid rgb(0, 0, 0); font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="266" /></span></div>
<span id="docs-internal-guid-80366f62-694a-9cd3-718d-d5bae03a49fa">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This is especially useful in real-time scenarios (for
example reading from a message bus). It’s one of those things that when we look
back seems pretty dumb that it wasn’t there from the start. It actually makes a
lot of sense, so we went ahead and made this the default behavior.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Streaming
results (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
When we launched streaming in Pentaho 8.0 we focused on the
processing piece. We could launch the sub transformation but we could not get
results back. Now we have the ability to define which step on the
sub-transformation will send back the results to follow the rest of the flow.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-dfe99cc2-694a-c1e3-af52-ee7c8a26d6e2"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-dfe99cc2-694a-c1e3-af52-ee7c8a26d6e2"><img alt="Kafka Consumer" height="243" src="https://lh4.googleusercontent.com/uUmqvbqYW9QfCbM8dwGn7BU058tUyHhFmcj922JVfLnsjk4PKECbxbce7dcEcVdLIQFR9crScijFQDEnhkNgRbKPTglzPA3X7L2xBNv__xWKXIwR1lHJ6Iw23o5_yBGKQB-P6xvxuDZoss4Zuw" style="border: 0pt solid rgb(0, 0, 0); font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="241" /></span></div>
<span id="docs-internal-guid-dfe99cc2-694a-c1e3-af52-ee7c8a26d6e2">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Why is this important? Because of what comes next…<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-top: 2.0pt; mso-pagination: widow-orphan lines-together; page-break-after: avoid;">
<span style="color: #1f3863; font-size: 12.0pt;">Streaming
Dataservices (EE/CE)<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
There’s a new option new option to run data service in
streaming mode. This will allow the consumers (on this case CTools Dashboards)
to get streaming data from this dataservice.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-53575cd2-694a-ed16-25dd-0b9be625345b"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-53575cd2-694a-ed16-25dd-0b9be625345b"><img height="110" src="https://lh6.googleusercontent.com/Ps_Q5u48MyZpq80bAOfgwi3Dr09PUjM36CeRV9YhRd2dD6ITbg1VPZ7AmZ8zitPV0NKGkUtV61-DfcJMyjIJTmQNEy8nZZEPnq7zvIYnKGphUlDCITKY4m4nGSQN7dit9HwUYa6XCg5_WWANLw" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="301" /></span></div>
<span id="docs-internal-guid-53575cd2-694a-ed16-25dd-0b9be625345b">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Once defined, we can test these options within the test
dataservices page and see the results as they come. <o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-4fcaf3f9-694b-0830-89e4-ff0732f70ecd"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-4fcaf3f9-694b-0830-89e4-ff0732f70ecd"><img height="177" src="https://lh4.googleusercontent.com/qtMuIMdUv3WJLrDfSxBhuqn_prOq23XBI_YTu0YtxH4Wg1T1UTVTb_Nd5jGll6XMc2vFRe404OIj0SpGGk6RcrypxLdp7T44L_Vd6w2wZs0sakAgRimFoJtUxt00NRc15Vl124rc3CYQyVbpjQ" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="302" /></span></div>
<span id="docs-internal-guid-4fcaf3f9-694b-0830-89e4-ff0732f70ecd">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This screen exposes the functionality as it would be called
from a client. It’s important to know that the windows that we define here are <i>not</i>
the same as the ones we defined for the micro batching service. The window
properties are the following:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Window Size – The number of rows that a window
will have (row based), or the time frame that we want to capture new rows to a
window (time based).<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Every - Number of rows (row based), or
milliseconds (time based) that should elapse before creating a new window.<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Limit – Maximum number of milliseconds (row
based) or rows (time based) which will be used to wait for a new window to be
generated.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>CTools
and Streaming Visualizations (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
We took a holistic approach to this feature. We want to make
sure we can have a real time / streaming dashboard leveraging what was set up
before. And this is where the CTools come in. There’s a new datasource in CDE
available to connect to streaming dataservices:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-a7b2f30e-694b-28d5-c0b8-23a9ddd3ae55"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-a7b2f30e-694b-28d5-c0b8-23a9ddd3ae55"><img height="124" src="https://lh3.googleusercontent.com/Nc7vrLQAVpFs2EzDXg_DwGCHa1teXygrXwngKXFu0NbcTGkhyQZ4I8niC-5o2ckCKAj4gV8dbJuUqIau3iI5DByzWLubxARhcn05F7a4X9Nm0lSRIypdhtkOA35zp7OsVgGD23xiDweQprdhBw" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="303" /></span></div>
<span id="docs-internal-guid-a7b2f30e-694b-28d5-c0b8-23a9ddd3ae55">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Then the configuration of the component will select the kind
of query we want – Time or number of records base, window size, frequency and
limit. This gives us a good control for a lot of use cases.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-6bf27dcf-694b-4900-3f44-519a3d4b30b1"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-6bf27dcf-694b-4900-3f44-519a3d4b30b1"><img height="320" src="https://lh6.googleusercontent.com/XtegxhVGJwhbAizh6yTe0oEK19PtToMcFcQhjzX__3-qXSjuwW4BB424nZA3mnFaPO1mBrXvse6YFnKYikQoxzus8RKyqahYoJfqZUwyjNIN9_BHh_6ZgywMbIbiiSa3Tdgr9sPfx8Oj_nbYbA" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="366" /></span></div>
<span id="docs-internal-guid-6bf27dcf-694b-4900-3f44-519a3d4b30b1">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This will allow us to then connect to a component the usual
way. While this will probably be more relevant for components like tables and
charts, ultimately all of them will work.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
It is possible to achieve a level of multi-tenancy by
passing a user name parameter from the PUC session (via CDE) to the
transformation as a data services push-down parameter. This will enable
restriction of the data viewed on a user by user basis<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
One important note is that the CTools streaming
visualizations do not yet operate on a ‘push’ paradigm – this is on the current
roadmap. In 8.1, the visualizations poll the streaming data service on a
constant interval which has a lower refresh limit of 1 second. But then again…
if you’re doing a dashboard of this types and need a refresh of 1 second, you’re
definitely doing something wrong…<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<h3 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Time
Series Visualizations (EE/CE)<o:p></o:p></b></h3>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
One of the biggest use cases for streaming, from a
visualization perspective, is time series. We improved the support for CCC for
timeseries line charts, so now data trends over time will be shown without
needing workarounds.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This applies not only to CTools but also to Analyzer<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<span id="docs-internal-guid-30987482-694b-779b-878a-dc60d31fbcce"></span><br />
<div style="text-align: center;">
<span id="docs-internal-guid-30987482-694b-779b-878a-dc60d31fbcce"><img height="208" src="https://lh6.googleusercontent.com/Ffh35UvuOl6DNIVJYYp2FYRW0frH4gEot7ByjGimVNKaXVbs5W8pDP2GV-HGqRgpbP5aXJnW3LHZJgBuh7f2QeE4zT7RvBI0msdKdyJFDqjjk51odpaHqxe5Z6FRvLdZ5YpLzimoT7G7c4iu1w" style="border: none; font-family: Calibri; font-size: 11pt; transform: rotate(0rad); white-space: pre-wrap;" width="303" /></span></div>
<span id="docs-internal-guid-30987482-694b-779b-878a-dc60d31fbcce">
</span>
<br />
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<h2 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
Data
Exploration Tool Updates (EE)<b><o:p></o:p></b></h2>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
We’re keeping on our path of improving our Data Exploration
Tool. It’s no secret that we want to make it feature complete so that it can
become <i>the</i> standard data analysis tool for the entire portfolio.<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
This time we worked on adding filters to the Stream view. <o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<span style="font-family: "calibri"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><img height="84" src="https://lh5.googleusercontent.com/oFL8DbMZK8lkLMsh_sHg6AkeAggzK7FubDimunknEsNxQ0LzU7YfLzvtafU1rwb9BC3sg2tgIOqTJetjJyz9zlgFVcV1NJXrWj9Exf--FTDik5sXGKKo7CdOwW2MqLJNeWVH2hUAEaIKYwmVPQ" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="166" /></span><span style="font-family: "calibri"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"> </span><span style="font-family: "calibri"; font-size: 11pt; vertical-align: baseline; white-space: pre-wrap;"><img height="228" src="https://lh6.googleusercontent.com/e2Q5Ckz4vCo31JqD9-wl1D0kAcsZxJSQbu4_3AsghiTgv21QXiZ8Ohp-9Rj88CnQM2xUyIMCPVJweAp-ys5gsP2_WH6ea40_cIepD_POhV5xa6HFCtvTMSnUZsIDz3Apei-QhNXJeIjvhJV5Rg" style="-webkit-transform: rotate(0.00rad); border: none; transform: rotate(0.00rad);" width="302" /></span><span style="mso-spacerun: yes;"> </span><o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
We’ll keep improving this. Next on the queue, hopefully,
will be filters on the model view and date filters!<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<h2 style="line-height: 115%; margin-bottom: 4.0pt; mso-pagination: lines-together;">
<b>Additional
Updates<o:p></o:p></b></h2>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
As usual, there were several additional updates that did not
make it to my highlights above. So for the sake of your time and not creating a
100 page blog – here are even more updates in Pentaho 8.1. <o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Additional updates:<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Salesforce connector API update (API version 41)<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Splunk connection updated to version 7<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Mongo version updated to 3.6.3 driver
(supporting 3.4 and 3.6)<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Cassandra version updated to support version 3.1
and Datastax 5.1<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->PDI repository browser performance updates,
including lazy loading<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Improvements on the Text and Hadoop file
outputs, including limit and control file handling<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Improved logging by removing auto-refresh from
the kettle logging servlet<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Admin can empty trash folder of other users on
PUC<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Clear button in PDI step search in spoon<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Override JDBC driver class and URL for a
connection<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Suppressed the Pentaho ‘session expired’ pop-up
on SSO scenarios, redirecting to the proper login page<o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l3 level1 lfo4; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="font-family: "noto sans symbols"; mso-bidi-font-family: "Noto Sans Symbols"; mso-fareast-font-family: "Noto Sans Symbols";"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]-->Included the possibility to schedule generation
of reports with a timestamp to avoid overwriting content<o:p></o:p></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
In summary (and wearing my marketing hat) with Pentaho 8.1
you can: <o:p></o:p></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l5 level1 lfo6; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="color: #616161; font-family: "noto sans symbols"; font-size: 10.0pt;"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b><span style="color: #616161;">Deploy in hybrid
and multi-cloud environments </span></b><span style="color: #616161;">with
comprehensive support for Google Cloud Platform, Microsoft Azure and AWS for
both data integration and analytics<b><o:p></o:p></b></span></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l5 level1 lfo6; tab-stops: .25in .5in; text-indent: -.25in;">
<!--[if !supportLists]--><span style="color: #616161; font-family: "noto sans symbols"; font-size: 10.0pt;"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b><span style="color: #616161;">Connect, process
and visualize streaming data, </span></b><span style="color: #616161;">from<b> </b>MQTT,
JMS, and IBM MQ message queues and gain insights from time series
visualizations<o:p></o:p></span></div>
<div class="MsoNormal" style="margin-left: .5in; mso-list: l5 level1 lfo6; tab-stops: .25in .5in; text-indent: -.25in;">
<a href="https://www.blogger.com/null" name="h.gjdgxs"></a><!--[if !supportLists]--><span style="color: #616161; font-family: "noto sans symbols"; font-size: 10.0pt;"><span style="mso-list: Ignore;">●<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><b><span style="color: #616161;">Get better
platform performance and increase user productivity </span></b><span style="color: #616161;">with improved logging, additional lineage information,
and faster repository access<o:p></o:p></span></div>
<div class="MsoNormal">
<br />
<br />
<br />
<h2 style="line-height: 27.6px; margin-bottom: 4pt;">
<b>Download it</b></h2>
<br />
<span style="font-family: inherit;"><a href="https://www.hitachivantara.com/en-us/products/big-data-integration-analytics/pentaho-trial-download.html?source=pedroalves-bi" style="background-color: white; display: inline; outline: none; text-align: justify; text-decoration-line: none; transition: color 0.3s;">Go get your Enterprise Edition or trial version from the usual places</a></span><br />
<span style="font-family: inherit;"><br style="background-color: white; text-align: justify;" /><span style="background-color: white; text-align: justify;">For CE, you can find it </span><a href="https://community.hds.com/docs/DOC-1009931-downloads" style="background-color: white; display: inline; outline: none; text-align: justify; text-decoration-line: none; transition: color 0.3s;">on the community home</a><span style="background-color: white; text-align: justify;">!</span></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
Pedro<o:p></o:p></div>
<br />Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com12tag:blogger.com,1999:blog-6644329693530300467.post-17757856036046666242017-11-15T18:20:00.000+00:002017-11-15T18:47:18.506+00:00Pentaho 8 is now available!<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.pentaho.com/product/version-8-0"><img border="0" data-original-height="245" data-original-width="800" height="195" src="https://community.hds.com/servlet/JiveServlet/downloadImage/102-1010503-3-23328/1600-490/17-152-8.0-launch-community-v1.jpg" width="640" /></a></div>
<br />
<br />
I recently wrote about <a href="http://pedroalves-bi.blogspot.pt/2017/10/pentaho80.html">everything you needed to know about Pentaho 8</a>. And now is available! <a href="http://www.pentaho.com/product/version-8-0">Go get your Enterprise Edition or trial version from the usual places</a><br />
<br />
For CE, you can find it <a href="https://community.hds.com/docs/DOC-1009931-downloads">on the new community home</a>!<br />
<br />
<br />
Enjoy!<br />
<br />
<br />
-pedroPedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com4tag:blogger.com,1999:blog-6644329693530300467.post-6818224915098668842017-11-09T11:22:00.000+00:002017-11-09T11:22:17.253+00:00A new collaboration space<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2IscsuEu72Lq-JBMkHaXmlpDFjJ_CM5gbkE3Tw7PFBcRO9edOeSa9MBy1qIluk0MO2YroUjQGt4COWcf9J83lWWOwCXPKXGHH97rj_9GXEXmEJUcDgNEWjhAHih6VjOGUWtDtoqcqH44/s1600/newForums.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="657" data-original-width="1146" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg2IscsuEu72Lq-JBMkHaXmlpDFjJ_CM5gbkE3Tw7PFBcRO9edOeSa9MBy1qIluk0MO2YroUjQGt4COWcf9J83lWWOwCXPKXGHH97rj_9GXEXmEJUcDgNEWjhAHih6VjOGUWtDtoqcqH44/s400/newForums.png" width="400" /></a></div>
<br />
With the move to Hitachi Vantara we're not letting the community go away - exactly on the contrary. And one of the first things is trying to give the community a new home, in here: <a href="http://community.pentaho.com/">http://community.pentaho.com</a><br />
<br />
<br />
We're trying to gather people from the forums, user groups, whatever, and give a better and more modern collaboration space. This space will continue open, also because the content is extremely value, so the ultimate decision is yours.<br />
<br />
Your mission, should you choose/decide to accept it, is to register and try this new home. Counting on your help to make it a better space<br />
<br />
<br />
See you in <a href="http://community.pentaho.com/">http://community.pentaho.com</a><br />
<br />
<br />
Cheers!<br />
<br />
<br />
-pedro<br />
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com2tag:blogger.com,1999:blog-6644329693530300467.post-11866758111942152032017-10-26T15:43:00.001+01:002017-10-26T15:43:19.502+01:00Announcing Pentaho 8.0 - Coming in November to a theater near you!<br />
<h2>
Pentaho 8!</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhECpFD4Yfbeped4y5p8rRaxkaWTV0aPq7gHvtBxiwZBGGqELuHr4wK1XlrkphkU27oi3iNg1o8A4lrNNanBEe6ywgXR5TfTXdzOGi0bahlUCAo-lOghlfJhSE8Exbx3cM0qUp8v6vJong/s1600/announce.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="559" data-original-width="1066" height="208" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhECpFD4Yfbeped4y5p8rRaxkaWTV0aPq7gHvtBxiwZBGGqELuHr4wK1XlrkphkU27oi3iNg1o8A4lrNNanBEe6ywgXR5TfTXdzOGi0bahlUCAo-lOghlfJhSE8Exbx3cM0qUp8v6vJong/s400/announce.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div>
<br /></div>
<h3>
The first of a new Era</h3>
<div>
Wow - time flies... Another <a href="https://www.pentahoworld.com/">Pentaho World</a> this week, and another blog post announcing another release. This time... <a href="http://www.pentaho.com/product/version-8-0">the best release ever</a>! ;)</div>
<div>
<br /></div>
<div>
This is our first Pentaho product announcement since we became <a href="https://www.hitachivantara.com/en-us/home.html">Hitachi Vantara</a> - and you'll see that some synergies are already appearing. And as I said before, again and again... the Community Edition is still around! We're not kidding - we're here to rule the world and we know it's though an open source core strategy that we'll get there :)</div>
<div>
<br /></div>
<h3>
Pentaho 8.0 In a nutshell</h3>
<div>
Ok, let's get on with this cause there's a lot of people at the bar calling me to have a drink. And I know my priorities! </div>
<div>
<br /></div>
<div>
<ul>
<li>Platform and Scalability</li>
<ul>
<li>Worker Nodes</li>
<li>New theme</li>
</ul>
<li>Data Integration</li>
<ul>
<li>Streaming support!</li>
<li>Run configurations for Jobs</li>
<li>Filters in Data Explorer</li>
<li>New Open / Save experience</li>
</ul>
<li>Big Data</li>
<ul>
<li>Improvements on AEL</li>
<li>Big Data File Formats - Avro and Parquet</li>
<li>Big Data Security - Support for Knox</li>
<li>VFS improvements for Hadoop Clusters</li>
</ul>
<li>Others</li>
<ul>
<li>Ops Mart for Oracle, MySQL, SQL Server</li>
<li>Platform password security improvements</li>
<li>PDI mavenization</li>
<li>Documentation changes on help.pentaho.com</li>
<li>Feature Removals:</li>
<ul>
<li>Analyzer on MongoDB</li>
<li>Mobile Plug-in (Deprecated in 7.1)</li>
</ul>
</ul>
</ul>
</div>
<div>
<br /></div>
<div>
Is it done? Can I go now? No?.... damn, ok, now on to further details...</div>
<div>
<br /></div>
<h3>
Platform and Scalability</h3>
<h4>
Worker Nodes (EE)</h4>
<div>
This is big. I never liked the way we handled scalability in PDI. Having the ETL designer responsible for manually defining the slave server in advance, having to control the flow of each execution, praying for things not to go down... nah! Also, why ETL only? What about all the other components of the stack?</div>
<div>
<br /></div>
<div>
So a couple of years ago, after getting info from a bunch of people I submitted a design document with a proposal for this:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkJtc9QztRmg2EZkUVt7Jsdq6wze5C9y7ctt9OoFbph9Kt_zug6bSSzbCjY0BvYG9rBJTg915xOykCb7c6nu-ufFGk8jW6AQPNpWfYcphc0ysUTnYVgqxylpU4Vvk_tYD8ymhLuYSykqM/s1600/02-DesignDoc_WorkerNodes+2017-10-24+10-28-49.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="519" data-original-width="464" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgkJtc9QztRmg2EZkUVt7Jsdq6wze5C9y7ctt9OoFbph9Kt_zug6bSSzbCjY0BvYG9rBJTg915xOykCb7c6nu-ufFGk8jW6AQPNpWfYcphc0ysUTnYVgqxylpU4Vvk_tYD8ymhLuYSykqM/s400/02-DesignDoc_WorkerNodes+2017-10-24+10-28-49.png" width="357" /></a></div>
<div>
This was way before I knew the term "worker nodes" was actually not original... but hey, they're nodes, they do work, and I'm bad with names, so there's that... :p</div>
<div>
<br /></div>
<div>
It took time to get to this point, not because we didn't think this was important, but because of the underlying order of execution; We couldn't do this without merging the servers, without changing the way we handle the repository, without having AEL (the Adaptive Execution Layer). Now we got to it!</div>
<div>
<br /></div>
<div>
Fortunately, we have an engineering team that can execute things properly! They took my original design, took a look at it, laughed at me, threw me out of the room and came up with the proper way of doing things. Here's the high-level description:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEha52uBlt8Ukk2pCB2lmiNB7vFN2KueEmHgpf6wMwF1L6SwKqpr7bzLoumzLAGn4LGFIg2wZVOH7xJxUV3HGmHYqNz-dDiR81M4qXafsE3ekMERp5426zKn7HYIC8SZ5z7I6SWcpobxdQE/s1600/03-WorkerNodes.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="378" data-original-width="760" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEha52uBlt8Ukk2pCB2lmiNB7vFN2KueEmHgpf6wMwF1L6SwKqpr7bzLoumzLAGn4LGFIg2wZVOH7xJxUV3HGmHYqNz-dDiR81M4qXafsE3ekMERp5426zKn7HYIC8SZ5z7I6SWcpobxdQE/s400/03-WorkerNodes.png" width="400" /></a></div>
<div>
<br /></div>
<div>
This is where I mentioned that we are already leveraging Hitachi Vantara resources. We are using <a href="https://community.hds.com/community/developer-network/hitachi-content-intelligence">Lumada Foundry</a> for worker nodes. Foundry is a platform for rapid development of service-based applications delivering the management of containers, communications, security, and monitoring toward creating enterprise products/applications, leveraging technology like docker, mesos, marathon, etc. More on this later, as it's something we'll be talking a lot more about...</div>
<div>
<br /></div>
<div>
Here's some of the features</div>
<div>
<div>
<ul>
<li>Deploy consistently in physical, virtual and cloud environments </li>
<li>Scale and load balance services , helping to deal with peaks and limited time-windows, allocate the resources that are needed.</li>
<li>Hybrid deployments can be used to distribute load, even when the on-premise resources are not sufficient, scaling out into the Cloud is possible to provide more resources. </li>
</ul>
</div>
</div>
<div>
So, how does this work in practice? Once you have a Pentaho Server installed, you can configure it to connect to the cluster of Pentaho Worker nodes. From that point on - things will work! No need to configure access to repositories, accesses, funky stuff. You only need to say "Execute at scale" and if the worker nodes are there, it's where things will be executed. Obviously, the "things will work" will have to obey the normal rules of clustered execution, for instance, don't expect a random node on the cluster to magically find out your <i>file:///c:/my computer/personal files/my mom's excel file.xls</i>.... :/</div>
<div>
<br /></div>
<div>
So what scenarios will this benefit the most? A lot! Now your server will not be bogged down executing a bunch of jobs and transformations as they will be handed out for execution in one of the nodes.</div>
<div>
<br /></div>
<div>
This does require some degree of control, because there may be cases where you <i>don't</i> want remote execution (for instance, a transformation to feed a dashboard). This is where <i>Run Configurations</i> come into play. Also important to note that even though the biggest benefits of this will be ETL work, this concept is for any kind of execution.</div>
<div>
<br /></div>
<div>
This a major part of the work we're doing with the Hitachi Vantara team; By leveraging Foundry we'll be able to do huge improvements on areas we've been wanting to tackle for a while but never were able to properly address on our own: better monitoring, improving lifecycle management and active-active HA, among others. In 8.0 we leapfrogged in this worker nodes story, and we expect much more going forward!</div>
<div>
<br /></div>
<div>
<h4>
New Theme - Ruby (EE/CE)</h4>
</div>
<div>
One of the things you'll notice is that we have a new theme that reflects the Hitachi Vantara colors. The new theme is the default on new installations (not for upgrades) and the others are still available</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-O9gAVvCEFLSOAftNmRZpJz8lSCNBE_xcB-VtCHzupcWDF35wIVwFkiZuFylHaLySsimjOMnkewQ5NUrYzptidOwu7Jqi027KD4f2iQJIFTvFTxlYD_xQ4vsjwjUSmThD3u-InCTnjsI/s1600/ruby.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="664" data-original-width="1237" height="213" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh-O9gAVvCEFLSOAftNmRZpJz8lSCNBE_xcB-VtCHzupcWDF35wIVwFkiZuFylHaLySsimjOMnkewQ5NUrYzptidOwu7Jqi027KD4f2iQJIFTvFTxlYD_xQ4vsjwjUSmThD3u-InCTnjsI/s400/ruby.png" width="400" /></a></div>
<div>
<br /></div>
<div>
<h3>
Data Integration</h3>
<h4>
Streaming Support: Kafka (EE/CE)</h4>
<div>
In Pentaho 8.0 we're introducing proper streaming support in PDI! In case you're thinking <i>"hum... but don't we already have a bunch of steps for streaming datasources? JMS, MQTT, etc?"</i> you're not wrong. But the problem is that PDI is a micro batching engine, and these streaming protocols introduce issues that can't be solved with the current approach. Just think about it - a streaming datasource requires an always running transformation, and in PDI execution all steps run in different threads while the data pipeline is being processed; There are cases, when something goes wrong, where we don't have the ability to do proper error processing. It's simply not as simple as a database query or any other call where we get a finite and well known amount of data.</div>
<div>
<br /></div>
<div>
So we took a different approach - somewhat similar to sub-transformations but not quite... First of all, you'll see a new section in PDI:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBixzQPMd5HC2ynp3_keDuJnudDHms5UJIX89D-qXBXKoBd4sdQRUDATARbLMy1LYmeC6mCTldrDmKuCXsS-9mTZ7-OXQm8oDy2wYKiRQcNUge3UJ2PahsyW1VjB2ze4genBRZ9mR5Big/s1600/pdi-streaming.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="442" data-original-width="342" height="320" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgBixzQPMd5HC2ynp3_keDuJnudDHms5UJIX89D-qXBXKoBd4sdQRUDATARbLMy1LYmeC6mCTldrDmKuCXsS-9mTZ7-OXQm8oDy2wYKiRQcNUge3UJ2PahsyW1VjB2ze4genBRZ9mR5Big/s320/pdi-streaming.png" width="246" /></a></div>
<div>
<br /></div>
<div>
Kafka is the one that was prioritized as being the most important for now, but this will actually be something that will be extended for other streaming sources.</div>
<div>
<br /></div>
<div>
The secret here is on the <i>Kafka Consumer</i> step:</div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaf_Iru_zoZ5L93YOgb5kqbEWCE3TEwNiB38A1fxZYLT_OgwRctC1X6KvWEnGo1mljfVW4p3vOqidlv9gPLs4C2dyKOu6fUNzJssn8DNJEFYGBV0Z6XjFZtIu5CC57blbYv1UCmvz31Jk/s1600/KafkaConsumer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="663" data-original-width="1020" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhaf_Iru_zoZ5L93YOgb5kqbEWCE3TEwNiB38A1fxZYLT_OgwRctC1X6KvWEnGo1mljfVW4p3vOqidlv9gPLs4C2dyKOu6fUNzJssn8DNJEFYGBV0Z6XjFZtIu5CC57blbYv1UCmvz31Jk/s400/KafkaConsumer.png" width="400" /></a></div>
<div>
<br /></div>
<div>
The highlighted tabs should be generic for pretty much all the steps, and the <i>Batch</i> is what controls the flow. So what we did was instead of having an always running transformation at the top level, we break the input data into chunks - either by number of records or duration and the second transformation takes that input, the fields structure and does a normal execution. In here, the abort step was also improved to give you more control the flow of this execution. This is actually something that's been a long standing request from the community - we can now specify if we want to abort with error or without, having an extra ability to control the flow of our ETL.</div>
<div>
<br /></div>
<div>
Here's an example of this thing put together:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_uj70NCbFUi5DEuDV8EqL6Fm-LIsOatZi1QJPlj7pt6y0jS7Scugp0gbLs1z-goTzO3ipZpSFQHxGhWE-JBxr3SZ4mMWBkt0KklJN5sJlyBzHp_gdOpxixjwI21cMnUo-avLSjLbT6uU/s1600/streamingdiagram.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="391" data-original-width="779" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi_uj70NCbFUi5DEuDV8EqL6Fm-LIsOatZi1QJPlj7pt6y0jS7Scugp0gbLs1z-goTzO3ipZpSFQHxGhWE-JBxr3SZ4mMWBkt0KklJN5sJlyBzHp_gdOpxixjwI21cMnUo-avLSjLbT6uU/s400/streamingdiagram.png" width="400" /></a></div>
<div>
Now, even more interesting that that is that this also works in AEL (our Adaptive Execution Layer, <a href="http://pedroalves-bi.blogspot.pt/2017/05/pentaho-7.1.html">introduced in Pentaho 7.1</a>), so when you run this on a cluster you'll get spark native kafka support being executed at scale, which is really nice...</div>
<div>
<br /></div>
<div>
Like I mentioned before, moving forward you'll see more developments here, namely:</div>
<div>
<ul>
<li>More streaming steps, and currently MQTT seems the best candidate for the short term</li>
<li>(and my favorite) Developer's documentation with a concrete example so that it's easy for anyone on the community to develop (and hopefully submit) their own implementations without having to worry about the 90% of the stuff that's common to all of them</li>
</ul>
<h4>
New Open / Save experience (EE/CE)</h4>
</div>
<div>
In <a href="http://pedroalves-bi.blogspot.pt/2016/10/pentaho-7.0.html">Pentaho 7.0</a> we merged the servers (no more that nonsense of having a distinct "BA Server" and a "DI Server") and introduced the unified <i>Pentaho Server</i> with a new and great looking experience to connect to it:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://2.bp.blogspot.com/npk1xLNWVAJBc1WfejfGVLGRnI1L3NbReHXgL0n5PyZSdHp5Y7Kp8OkfZERAxD98oVBacjIfiSHuR2nc25staD_Vw8NnHWV8d9ld8CNNVE1LjEisEyte4dJ7-ef63LUYJj1tO0GQ_HXyDtC-0Q" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="482" data-original-width="750" height="256" src="https://2.bp.blogspot.com/npk1xLNWVAJBc1WfejfGVLGRnI1L3NbReHXgL0n5PyZSdHp5Y7Kp8OkfZERAxD98oVBacjIfiSHuR2nc25staD_Vw8NnHWV8d9ld8CNNVE1LjEisEyte4dJ7-ef63LUYJj1tO0GQ_HXyDtC-0Q" width="400" /></a></div>
<div>
<br /></div>
<div>
but then I clicked on <i>Open file from repository</i> and felt sick... That thing was absolutely horrible and painfully slow. We were finally able to do something about that! Now the experience is ... well... <i>slightly</i> better (as in, I don't feel like throwing up anymore!):</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK-GYlkTlaN6G9InLTQk8PtFdTl29p_SyD9-wNFD-AcNFCRrL8u3s4dzK87KEK80cuHHYqiwKq4ARfnzUlcqTteUNFsscbSfmzE6M_0mLoM6I0f4dBK6b2yQyIRUi3aN7xVGlkD0AY0xI/s1600/pdi-opensave.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="663" data-original-width="1159" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhK-GYlkTlaN6G9InLTQk8PtFdTl29p_SyD9-wNFD-AcNFCRrL8u3s4dzK87KEK80cuHHYqiwKq4ARfnzUlcqTteUNFsscbSfmzE6M_0mLoM6I0f4dBK6b2yQyIRUi3aN7xVGlkD0AY0xI/s400/pdi-opensave.png" width="400" /></a></div>
<div>
<br /></div>
<div>
A bit better, no? :) Also with search capabilities and all the kind of stuff that you've been expecting from a dialog like this on the past 10 years! Same for the save experience.</div>
<div>
<br /></div>
<div>
This is another small but IMO always important step in unifying the user experience and work towards a product that gets progressively more pleasant to use. It's a never-ending journey but that's not an excuse not to take it.</div>
<div>
<br /></div>
<h4>
Filters in Data Explorer (EE)</h4>
<div>
Now that I was able to open my transformation, I can show some of the improvements that we did on our <i>Data Explorer</i> experience in PDI. We now support the first set of filters and actions! This one is easy to show but extremely powerful to use.</div>
<div>
<br /></div>
<div>
Here's filters - depending on the data type you'll have a few options, like excluding nulls, equals, greater/lesser than and a few others. Like mentioned, others will come with time. </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqLBxXNQ-j2lkIXKq1QBEZbQaeJiZzQWOCAjbKEz5SpMP_BqrAvg4eLqQkcm2iQRqjipzD5MDeiXZ_6DN-1nPiEgZJsqbs1uo05So3MgTFSNWjiM-ot9Nn0byHX12Iyd0izZ_g9rzVaeI/s1600/filters.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="670" data-original-width="1167" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjqLBxXNQ-j2lkIXKq1QBEZbQaeJiZzQWOCAjbKEz5SpMP_BqrAvg4eLqQkcm2iQRqjipzD5MDeiXZ_6DN-1nPiEgZJsqbs1uo05So3MgTFSNWjiM-ot9Nn0byHX12Iyd0izZ_g9rzVaeI/s400/filters.png" width="400" /></a></div>
<div>
<br /></div>
<div>
Also, while previous version only allowed for drill down, we can now do more operations on the visualizations.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirxFbG54jcWEwU25Kh-FZV8usUOJ92WQB2hNTo9RpIXVXwbgcL0jG7ZrUJ6927cK6aKdP2Xj0SkPuGytSjwkT4DGAvEC4UE3DjooGSsmwKwJOKYsPK_9LB_HxxhGH6y8mBckPRig2RjEQ/s1600/actions.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="670" data-original-width="1167" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEirxFbG54jcWEwU25Kh-FZV8usUOJ92WQB2hNTo9RpIXVXwbgcL0jG7ZrUJ6927cK6aKdP2Xj0SkPuGytSjwkT4DGAvEC4UE3DjooGSsmwKwJOKYsPK_9LB_HxxhGH6y8mBckPRig2RjEQ/s400/actions.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<h4>
Run configuration: Leveraging worker nodes and execute on server (EE/CE)</h4>
<div>
Now that we are connected to the repository, opened our transformation with a really nice experience and took benefit of these data exploration improvements to make sure our logic is spot on, we are ready to execute it to the server. </div>
<div>
<br /></div>
<div>
Now this is where the run configuration part comes in. I have my transformation, defined it, played with it, verified that really works as expected on my box. And now, I will want to make sure it also runs well on the server. What before was a very convoluted process, it's now much simplified.</div>
<div>
<br /></div>
<div>
What I do is define a new <i>Run Configuration</i>, like described in 7.1 for AEL, but with a little twist: I don't want it to use the spark engine; I want it to use the pentaho engine but on the server, not the one local to spoon:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGX-UpFjRbwNs0wTbrMIHFUtoXnjbymKL4qr__XHryROMJ9SGvHk6hoLAwb2BrPCCrzjVh70XdCaKYejpfPfa2Ghgx3TqZHAulVekY9n1knALOIfWaunks_sOqrKxunRzlfkGLxlxky8E/s1600/run_config.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="544" data-original-width="555" height="391" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGX-UpFjRbwNs0wTbrMIHFUtoXnjbymKL4qr__XHryROMJ9SGvHk6hoLAwb2BrPCCrzjVh70XdCaKYejpfPfa2Ghgx3TqZHAulVekY9n1knALOIfWaunks_sOqrKxunRzlfkGLxlxky8E/s400/run_config.png" width="400" /></a></div>
<div>
</div>
<div>
Now, what happens when I execute this selecting the Pentaho Server run configuration?</div>
<div>
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1nph4vXAVNyufBZCpRmRWZ9_C2Jak8bB3LafnBKIvBLxFXC57Sig2T6HizFoVOykJHNVZ5g8sXcNPZl39EvkD-dyFAwZtYxxFuyA4SHJIWp16_vOOib4E0vxCYKsNHNEM8Dih2giGtCU/s1600/run_config_dialog.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="571" data-original-width="660" height="345" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1nph4vXAVNyufBZCpRmRWZ9_C2Jak8bB3LafnBKIvBLxFXC57Sig2T6HizFoVOykJHNVZ5g8sXcNPZl39EvkD-dyFAwZtYxxFuyA4SHJIWp16_vOOib4E0vxCYKsNHNEM8Dih2giGtCU/s400/run_config_dialog.png" width="400" /></a></div>
<div>
<br /></div>
<div>
Yep, that!! \o/</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFDbrh0G00Cp5q9TxVM9OO4jengx28QRaljO6YHhJyUExayLnc3mA0FdvKRlrYiTz1tKKKVHWeJkZX3y47g7EgXlEOfbMmvHsqgKuPce5dLwKLeyLekCx5t0lpbeJsUre4GUAnQe_fAr8/s1600/executeOnServer.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="667" data-original-width="1164" height="228" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiFDbrh0G00Cp5q9TxVM9OO4jengx28QRaljO6YHhJyUExayLnc3mA0FdvKRlrYiTz1tKKKVHWeJkZX3y47g7EgXlEOfbMmvHsqgKuPce5dLwKLeyLekCx5t0lpbeJsUre4GUAnQe_fAr8/s400/executeOnServer.png" width="400" /></a></div>
<div>
<br /></div>
<div>
This screenshot shows PDI trigger the execution and my Pentaho Server console logging it's execution.</div>
<div>
<br /></div>
<div>
And if I had worker nodes configured, what I would see would be my Pentaho Server <i>automatically</i> dispatching the execution of my transformation to an available worker node! </div>
<div>
<br /></div>
<div>
This doesn't apply to the immediate execution only; We can now specify the run configuration on the job entry as well, allowing a full control of the flow of our more complex ETL</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXkVRnPr080A5J-C1lBrXHCsTuY06gb6E4a9KwfJjm02lkgM6681VtIn_NpUTjHWw2oHBucSv6t9BlWQub9d9AT_KLdAnU9WWzf-O58-O94YK32LmrzO_KwYnIJToq33DUC2IlaVKPW4k/s1600/jobentry.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="718" data-original-width="750" height="382" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiXkVRnPr080A5J-C1lBrXHCsTuY06gb6E4a9KwfJjm02lkgM6681VtIn_NpUTjHWw2oHBucSv6t9BlWQub9d9AT_KLdAnU9WWzf-O58-O94YK32LmrzO_KwYnIJToq33DUC2IlaVKPW4k/s400/jobentry.png" width="400" /></a></div>
<div>
<br /></div>
<h3>
Big Data</h3>
<h4>
Improvements on AEL (EE/CE <span style="font-weight: normal;">apart from the security bits</span>)</h4>
<div>
As expected, a lot of work was done on AEL. The biggest ones:</div>
<div>
<ul>
<li>Communicates with Pentaho client tools over WebSocket; does NOT require Zookeeper</li>
<li>Uses distro-specific Spark library</li>
<li>Enhanced Kerberos impersonation on client-side</li>
</ul>
</div>
<div>
<div>
This brings a bunch of benefits:</div>
<div>
<ul>
<li>Reduced number of steps to setup </li>
<li>Enable fail-over, load-balancing</li>
<li>Robust error and status reporting </li>
<li>Customization of Spark jobs (i.e. memory , settings)</li>
<li>Client to AEL connection can be secured</li>
<li>Kerberos impersonation from client tool </li>
</ul>
</div>
</div>
<div>
And not to mention performance improvements... One benchmark I saw that I found particularly impressive is that AEL is practically on pair with native spark execution! And <i>this</i> is impressive! Kudos for the team, just spectacular work!</div>
<h4>
Big Data File Formats - Avro and Parquet (EE/CE)</h4>
<div>
Big data platforms introduced various data formats to improve performance, compression and interoperability, and we added full support for these very popular big data formats: Avro and Parquet. Orc will come next.</div>
<div>
<br /></div>
<div>
When you run in AEL, these will also be natively interpreted by the engine, which adds a lot to the value of this.</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtaOqAJ0VY3xT76f66YE1dWbrEpNccEC1gNsnEFhNgr_pCtCCUJZ5iGQBPbPscPxsFXcDXYT-qDOhmkd_JPd0py7oehvp1gNRJW5VO7R8d0-IdOn1cAqXCCeTzi0ANbwJ61ZfO3KOfjg8/s1600/bigdataformats.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="670" data-original-width="1182" height="226" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhtaOqAJ0VY3xT76f66YE1dWbrEpNccEC1gNsnEFhNgr_pCtCCUJZ5iGQBPbPscPxsFXcDXYT-qDOhmkd_JPd0py7oehvp1gNRJW5VO7R8d0-IdOn1cAqXCCeTzi0ANbwJ61ZfO3KOfjg8/s400/bigdataformats.png" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
The old steps will still be available on the marketplace but we don't recommend using them.</div>
<div>
<br /></div>
<h4>
Big Data Security - Support for Knox</h4>
<div>
Knox provides perimeter security so that the enterprise can confidently extend Hadoop access to more of those new users while also maintaining compliance with enterprise security policies and used in some HortonWorks deployments. It is now supported on the Hadoop Clusters' definition if you enable the property <i>KETTLE_HADOOP_CLUSTER_GATEWAY_CONNECTION</i> on the kettle.properties file.</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5q6eRQthugEM5Hsc9eqzrsguTT2qR-pooNpGEIXxCXdz4mJZaXI0zcTU_mhM8_xh7Q7LXMJ20CdmNpBgg9Mm_raVc-teMgy6pnjdFezbMh5xzmphdeccfjBxFvvGUM9bJTYuhwQA-yFw/s1600/knox.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="317" data-original-width="314" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj5q6eRQthugEM5Hsc9eqzrsguTT2qR-pooNpGEIXxCXdz4mJZaXI0zcTU_mhM8_xh7Q7LXMJ20CdmNpBgg9Mm_raVc-teMgy6pnjdFezbMh5xzmphdeccfjBxFvvGUM9bJTYuhwQA-yFw/s1600/knox.png" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<h4>
VFS improvements for Hadoop Clusters (EE/CE)</h4>
<div>
In order to simplify the overall lifecycle of jobs and transformations we made the hadoop clusters available through VFS, on the format <i>hc://hadoop_cluster<hadoop cluster="">/</hadoop></i>. </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgD6fmkz7owb36G5fmENcfGGcGPQ06JlWTmfbtCCEywfEn5E3ymzRz1mKWVC3ljpx4HhASEI3hvgeHAXZy_nCZI5l59-vhGLZNQ6WUm0KCl_8XgeWlJu1iRkbLsTNG9216sVJiEyDx7AKQ/s1600/namedclusters.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="243" data-original-width="361" height="214" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgD6fmkz7owb36G5fmENcfGGcGPQ06JlWTmfbtCCEywfEn5E3ymzRz1mKWVC3ljpx4HhASEI3hvgeHAXZy_nCZI5l59-vhGLZNQ6WUm0KCl_8XgeWlJu1iRkbLsTNG9216sVJiEyDx7AKQ/s320/namedclusters.png" width="320" /></a></div>
<div>
<br /></div>
<h3>
Others</h3>
</div>
<div>
There are some other generic improvements worth noting</div>
<div>
<br /></div>
<h4>
Ops Marts extended support (EE)</h4>
<div>
Ops Mart now supports Oracle, MySQL and SQL Server. I can't really believe I'm still writing about this thing :(</div>
<div>
<h4>
PDI Mavenization (CE)</h4>
</div>
<div>
Now, this is actually nice! PDI is now fully mavenized. Go to <a href="https://github.com/pentaho/pentaho-kettle">https://github.com/pentaho/pentaho-kettle</a>, do a <i>mvn package</i> and you're done!!!</div>
<div>
<br />
<br />
-----------<br />
<br />
<br /></div>
<div>
<div class="MsoNormal">
<span lang="EN-US" style="font-family: "Times",serif; font-size: 11.0pt; mso-bidi-font-family: Times;">Pentaho 8 will be available to download mid-November.</span></div>
<div class="MsoNormal">
<span lang="EN-US" style="font-family: "Times",serif; font-size: 11.0pt; mso-bidi-font-family: Times;"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US" style="font-family: "Times",serif; font-size: 11.0pt; mso-bidi-font-family: Times;">Learn more about Pentaho 8.0 and a webinar here: </span><span lang="EN-US" style="mso-fareast-font-family: "Times New Roman";"><a href="http://www.pentaho.com/product/version-8-0" target="_blank"><span style="color: #954f72; font-family: "Arial",sans-serif; font-size: 10.0pt;">http://www.pentaho.com/product/version-8-0</span></a><o:p></o:p></span></div>
<div class="MsoNormal" style="mso-layout-grid-align: none; mso-pagination: none; text-autospace: none;">
<span lang="EN-US" style="font-family: "Times",serif; font-size: 11.0pt; mso-bidi-font-family: Times;">Also, you can get a glimpse of PentahoWorld this
week watching it live at: <a href="http://siliconangle.tv/pentaho-world-2017/">http://siliconangle.tv/pentaho-world-2017/</a><o:p></o:p></span></div>
<div class="MsoNormal" style="mso-layout-grid-align: none; mso-pagination: none; text-autospace: none;">
<br /></div>
<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
<o:PixelsPerInch>96</o:PixelsPerInch>
</o:OfficeDocumentSettings>
</xml><![endif]-->
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="false"
DefSemiHidden="false" DefQFormat="false" DefPriority="99"
LatentStyleCount="382">
<w:LsdException Locked="false" Priority="0" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 9"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="header"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footer"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index heading"/>
<w:LsdException Locked="false" Priority="35" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of figures"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope return"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="line number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="page number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of authorities"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="macro"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="toa heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 5"/>
<w:LsdException Locked="false" Priority="10" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Closing"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Signature"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Message Header"/>
<w:LsdException Locked="false" Priority="11" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Salutation"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Date"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Block Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="FollowedHyperlink"/>
<w:LsdException Locked="false" Priority="22" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Document Map"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Plain Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="E-mail Signature"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Top of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Bottom of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal (Web)"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Acronym"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Cite"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Code"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Definition"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Keyboard"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Preformatted"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Sample"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Typewriter"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Variable"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Table"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation subject"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="No List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Contemporary"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Elegant"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Professional"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Balloon Text"/>
<w:LsdException Locked="false" Priority="39" Name="Table Grid"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Theme"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 9"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" QFormat="true"
Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" QFormat="true"
Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" QFormat="true"
Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" QFormat="true"
Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" QFormat="true"
Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" QFormat="true"
Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" SemiHidden="true"
UnhideWhenUsed="true" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="TOC Heading"/>
<w:LsdException Locked="false" Priority="41" Name="Plain Table 1"/>
<w:LsdException Locked="false" Priority="42" Name="Plain Table 2"/>
<w:LsdException Locked="false" Priority="43" Name="Plain Table 3"/>
<w:LsdException Locked="false" Priority="44" Name="Plain Table 4"/>
<w:LsdException Locked="false" Priority="45" Name="Plain Table 5"/>
<w:LsdException Locked="false" Priority="40" Name="Grid Table Light"/>
<w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="46" Name="List Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Mention"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Smart Hyperlink"/>
</w:LatentStyles>
</xml><![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Arial;
panose-1:2 11 6 4 2 2 2 2 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536859905 -1073711037 9 0 511 0;}
@font-face
{font-family:Times;
panose-1:2 0 5 0 0 0 0 0 0 0;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536870145 1073786111 1 0 415 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
mso-themecolor:hyperlink;
text-decoration:underline;
text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:#954F72;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;
mso-header-margin:36.0pt;
mso-footer-margin:36.0pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
-->
</style>
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
</style>
<![endif]-->
<!--StartFragment-->
<!--EndFragment--><br />
<div class="MsoNormal" style="mso-layout-grid-align: none; mso-pagination: none; text-autospace: none;">
<span lang="EN-US" style="font-family: "Times",serif; font-size: 11.0pt; mso-bidi-font-family: Times;">Last but not See you in a few weeks at the <a href="https://it-novum.com/en/pcm17/">Pentaho Community meeting</a> in Mainz! <a href="https://it-novum.com/en/pcm17/">https://it-novum.com/en/pcm17/</a><o:p></o:p></span></div>
<br /></div>
<div>
That's it - I'm going to the bar!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
-pedro</div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com8tag:blogger.com,1999:blog-6644329693530300467.post-91415798536596048062017-09-19T17:01:00.000+01:002017-09-20T18:06:21.446+01:00Hello Hitachi Vantara!<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5mlBS4tB5nIvT1J1zIM-1peZo6CNSCUphhO1deh18QIC6wcsyooGq0VK6nzBkZzB2o9Xiq7BT5jrGsFSUcYet2O3hneWeMmygaYxztKYey13O3rnETKcAiUUoNw5ABFH-ZVXlnP8PLGU/s1600/cslogo.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="106" data-original-width="365" height="92" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi5mlBS4tB5nIvT1J1zIM-1peZo6CNSCUphhO1deh18QIC6wcsyooGq0VK6nzBkZzB2o9Xiq7BT5jrGsFSUcYet2O3hneWeMmygaYxztKYey13O3rnETKcAiUUoNw5ABFH-ZVXlnP8PLGU/s320/cslogo.png" width="320" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
Ok, I admit it – I am one of those people that actually likes changes and views it as an opportunity. Four years ago, <a href="http://pedroalves-bi.blogspot.pt/search?q=pentaho+joins+webdetails">I announced here that Webdetails joined Pentaho</a>. For the ones who don't know, Webdetails was the Portugese-based consulting company that then turned into Pentaho Portugal (and expanded from 20 people at the time to 60+), completely integrated into the Pentaho structure.<br />
<div>
<br /></div>
Two years ago, <a href="http://pedroalves-bi.blogspot.com/2015/02/big-news-today-hitachi-data-systems-hds.html">we announced that Pentaho was acquired by HDS</a>, becoming a Hitachi Group Company.<br />
<div>
<br /></div>
We have a new change today - and since I'm lazy (and in Vegas, for the Hitachi Next event, and would rather be at our party at the Mandalay Bay Beach than in my room writing this blog post!), I'll simply steal the same structure I used two years ago (when Pentaho was acquired) and get straight to the point! :p<br />
<h3>
Big news</h3>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0fNaabazE8a_1csv4NBUUNZI1Kpj3coGKzaa7vPT2U7-d3MHg0oiYbnXekDTdjxQkc68k6PfQKePmhf7JKgFeTj8Vtr5ZKF-jd42hWBBTQtkHFvlDCp1bUK4QdKGVZvBe9x7IBkrsv-s/s1600/17-148-Hitachi-NewCo-blog-v1.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="678" data-original-width="1600" height="135" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj0fNaabazE8a_1csv4NBUUNZI1Kpj3coGKzaa7vPT2U7-d3MHg0oiYbnXekDTdjxQkc68k6PfQKePmhf7JKgFeTj8Vtr5ZKF-jd42hWBBTQtkHFvlDCp1bUK4QdKGVZvBe9x7IBkrsv-s/s320/17-148-Hitachi-NewCo-blog-v1.jpg" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div>
An extremely big transformation has been taking place and materialized itself today, September 19, 2017. A new company is born. Meet: <b><a href="https://www.hitachivantara.com/">Hitachi Vantara</a></b></div>
<div>
<b><br /></b></div>
<div>
You may be asking yourselves: Can it possibly be a coincidence that the new company is launched on the exact same day I turn 40? Well, actually yes, a complete coincidence... :/</div>
<div>
<b><br /></b></div>
<div>
This new company unifies the mission and operations of Pentaho, Hitachi Data Systems and Hitachi Insight Group into a single business. More info in the <a href="http://www.pentaho.com/blog/pentaho-becomes-hitachi-vantara">Pentaho blog: Hitachi Vantara - Here's what it means</a></div>
<div>
<br /></div>
<h3>
What does this mean?</h3>
<br />
It has always been our goal to provide an offering that would allow customers to build their high value, data driven solutions. We were, I think, successful at doing that! And now we (Hitachi Vantara) want to take it to the next level, thus this transformation is needed: We're aiming higher - we want to not only to be the best at (big) data orchestration and analytics, we want to do so in this new IoT / social innovation ecosystem aiming to be the biggest player in the market.<br />
<br />
<br />
<div>
And this transformation will allow us to do that!</div>
<div>
<br /></div>
<h3>
What will change?</h3>
So that it's clear, Pentaho, as a product will continue to exist. Pentaho, as a company, is now Hitachi Vantara. <br />
<br />
<div>
And for Pentaho as a product, this gives us conditions we've never had to improve the product focusing on what we need to do best (big data orchestration and analytics) and leveraging from other groups in the company on areas that even though they weren't our core focus, people expect us to have. </div>
<div>
<br /></div>
<div>
Overall, we'll also improve the overall portfolio interoperability. While so far we've always tried to be completely agnostic, now we'll keep saying that but add a small detail: <i>But we have to work better with our stuff</i> - because we can make it happen! </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Community implications</h3>
<div>
This one is very easy!!! I'll just copy paste my previous answer - because it didn't change:</div>
<div>
<br /></div>
<blockquote class="tr_bq">
Throughout all the talks, our relationship and involvement with the community has always been one of the strong points of Pentaho, and seen with much interest.<br />
The relationship between the community and a commercial company exists because it’s mutually beneficial. In Pentaho’s case, the community gets access to software it otherwise couldn’t, and Pentaho gets access to an insane amount of resources that contribute to the project. Don’t believe me? Check the Pentaho Marketplace for the large number of submissions, Jira for all the bug reports and improvement suggestions we get out of all the real world tests, and discussions on the forums or on the several available email lists.<br />
Is anyone, in his or her right mind, willing to let all this go? Nah.<br />
Plus, not having a community would render my job obsolete, and no one wants that, right? (don’t answer, please!)</blockquote>
<br />
The difference? We wanna do this bigger, better and faster!<br />
<br />
<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
<o:PixelsPerInch>96</o:PixelsPerInch>
</o:OfficeDocumentSettings>
</xml><![endif]-->
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="false"
DefSemiHidden="false" DefQFormat="false" DefPriority="99"
LatentStyleCount="382">
<w:LsdException Locked="false" Priority="0" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 9"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="header"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footer"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index heading"/>
<w:LsdException Locked="false" Priority="35" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of figures"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope return"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="line number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="page number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of authorities"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="macro"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="toa heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 5"/>
<w:LsdException Locked="false" Priority="10" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Closing"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Signature"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Message Header"/>
<w:LsdException Locked="false" Priority="11" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Salutation"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Date"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Block Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="FollowedHyperlink"/>
<w:LsdException Locked="false" Priority="22" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Document Map"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Plain Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="E-mail Signature"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Top of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Bottom of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal (Web)"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Acronym"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Cite"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Code"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Definition"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Keyboard"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Preformatted"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Sample"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Typewriter"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Variable"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Table"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation subject"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="No List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Contemporary"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Elegant"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Professional"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Balloon Text"/>
<w:LsdException Locked="false" Priority="39" Name="Table Grid"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Theme"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 9"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" QFormat="true"
Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" QFormat="true"
Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" QFormat="true"
Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" QFormat="true"
Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" QFormat="true"
Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" QFormat="true"
Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" SemiHidden="true"
UnhideWhenUsed="true" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="TOC Heading"/>
<w:LsdException Locked="false" Priority="41" Name="Plain Table 1"/>
<w:LsdException Locked="false" Priority="42" Name="Plain Table 2"/>
<w:LsdException Locked="false" Priority="43" Name="Plain Table 3"/>
<w:LsdException Locked="false" Priority="44" Name="Plain Table 4"/>
<w:LsdException Locked="false" Priority="45" Name="Plain Table 5"/>
<w:LsdException Locked="false" Priority="40" Name="Grid Table Light"/>
<w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="46" Name="List Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Mention"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Smart Hyperlink"/>
</w:LatentStyles>
</xml><![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536870145 1073786111 1 0 415 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin:0cm;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;
mso-header-margin:36.0pt;
mso-footer-margin:36.0pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
</style>
<br />
-->
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
</style>
<![endif]-->
<!--StartFragment-->
<!--EndFragment--><div class="MsoNormal">
<span lang="EN-US" style="color: #646464; font-size: 11.0pt;">And
things are already moving in that direction. We are moving the Pentaho
Community page to the Hitachi Vantara communit site with some really col
interactive and social features. You can visit our new home here </span><span lang="EN-US" style="font-family: "times new roman" , serif; mso-fareast-font-family: "Times New Roman";"><a href="https://community.hds.com/community/products-and-solutions/pentaho"><span style="color: #954f72; font-family: "calibri" , sans-serif; font-size: 11.0pt;">https://community.hitachivantara.com/community/products-and-solutions/pentaho</span></a></span><span lang="EN-US" style="background: white; color: black; font-size: 11.0pt;">. I look forward to engaging with all of you on
this new site. </span><span lang="EN-US" style="font-family: "times new roman" , serif; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
<br />
<div>
<h3>
Will Hitachi Vantara shut down it's Pentaho CE edition / it's open source model?</h3>
<div>
I will, once again, repeat the previous answer:</div>
<div>
<br /></div>
<blockquote class="tr_bq">
Just in case the previous answer wasn’t clear enough, lemme spell it out with all the words: There are no plans of changing our opensource strategy or stop providing a CE edition to our community!<br />
Can that change in the future? Oh, absolutely yes! Just like it could have changed in the past. And when could it change? When it stops making sense; when it stops being mutually beneficial. And on that day, I’ll be the first one to suggest a change to our model.</blockquote>
<div>
<br /></div>
</div>
<div>
And speaking of which - <a href="https://it-novum.com/en/pcm17/">don't forget to register to PCM17</a>! It's going to be the best ever!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Cheers!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
-pedro </div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com9tag:blogger.com,1999:blog-6644329693530300467.post-84984017672803957412017-09-14T11:41:00.000+01:002017-09-20T18:06:01.863+01:00Pentaho Community Meeting 2017: exciting use cases & final Call for Papers<div class="MsoNormal" style="mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;">
<span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%;">Enjoyed your vacations? Good - now let's get back in business!</span></div>
<div class="MsoNormal" style="mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;">
<span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%;"><br />The Pentaho Community Meeting 2017 in Mainz, taking place from November
10-12, is approaching and more than 140 participants interested in BI and
Big Data are already on board. <br />
<br />
Many great speakers from all over the world will present their Pentaho
use cases, including data management and analysis at CERN, evaluation of
environmental data at the Technical University of Liberec and
administration of health information in Mozambique. And of course Matt
Casters, Pedro Alves and Jens Bleuel will introduce the latest features
in Pentaho.</span><span lang="EN-US" style="mso-ansi-language: EN-US;"><br />
</span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%;"><br />
<br />
<b style="mso-bidi-font-weight: normal;">The 10<sup>th</sup> jubilee
edition features many highlights:</b></span><span lang="EN-US" style="mso-ansi-language: EN-US;"><o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l1 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;">Hackathon and technical
presentations on FRI, Nov 10 <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l1 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;">Conference day on SAT, Nov
11
<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l1 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="DE" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="DE" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;">Dinner on SAT, Nov 11
<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l1 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;">Get-together and drinks on SAT, Nov
11 <o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l1 level1 lfo1; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;">Social event on SUN, Nov 12<o:p></o:p></span></div>
<div class="MsoNormal" style="mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;">
<span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%;"><br />
</span><span lang="EN-US" style="mso-ansi-language: EN-US;">See here the complete
</span><span lang="DE"><a href="https://it-novum.com/en/pcm17/#c5439" target="_blank"><span lang="EN-US" style="mso-ansi-language: EN-US;">agenda </span></a></span><span lang="EN-US" style="mso-ansi-language: EN-US;">with all presentations of the
business and technical track on the conference day. Food and drinks will
be provided. Highlight to the CERN use case (<a href="https://www.blogger.com/nterview%20with%20CERN,%20here%20is%20the%20link:%20https://it-novum.com/blog/cern-accelerates-its-business-computing-with-pentaho/">you can read a blog post on it here</a>)<br />
<br />
<b style="mso-bidi-font-weight: normal;">And don’t forget: you can
participate in the Call for Papers till September 30th! </b>Send your
Pentaho project to Jens Bleuel via the </span><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2873" target="_blank"><span lang="EN-US" style="color: #db0031; mso-ansi-language: EN-US;">contact form</span></a></span><span lang="EN-US" style="mso-ansi-language: EN-US;">.<br />
<br />
<b style="mso-bidi-font-weight: normal;"><span style="mso-spacerun: yes;"> </span>Some of the speakers: <o:p></o:p></b></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l0 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2880"><span lang="EN-US" style="color: #db0031; font-family: "arial" , sans-serif; font-size: 10.0pt;">Pedro Alves</span></a></span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;"> - Aka... me! All about Pentaho 8.0, which is a different way to say "hum, just put some random title, I'll figure out something later"<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l0 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2881"><span lang="EN-US" style="color: #db0031; font-family: "arial" , sans-serif; font-size: 10.0pt;">Dan Keeley</span></a></span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;"> - Data Pipelines -
Running PDI on AWS Lambda<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l0 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2882"><span lang="EN-US" style="color: #db0031; font-family: "arial" , sans-serif; font-size: 10.0pt;">Francesco Corti</span></a></span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;"> - Pentaho
8 Reporting for Java Developers<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l0 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2883"><span lang="EN-US" style="color: #db0031; font-family: "arial" , sans-serif; font-size: 10.0pt;">Pedro Vale</span></a></span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;"> - Machine Learning in PDI
- What's new in the Marketplace?<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l0 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";"> </span></span></span><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2884"><span lang="EN-US" style="color: #db0031; font-family: "arial" , sans-serif; font-size: 10.0pt;">Caio Moreno de Souza</span></a></span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;"> - Working
with Automated Machine Learning (AutoML) and Pentaho<o:p></o:p></span></div>
<div class="MsoNormal" style="line-height: normal; margin-bottom: .0001pt; margin-bottom: 0cm; margin-left: 32.25pt; margin-right: 0cm; margin-top: 0cm; mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; mso-list: l0 level1 lfo2; tab-stops: list 36.0pt; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="color: black; font-family: "symbol"; font-size: 10.0pt;"><span style="mso-list: Ignore;">·<span style="font: 7.0pt "Times New Roman";">
</span></span></span><!--[endif]--><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2885"><span lang="EN-US" style="color: #db0031; font-family: "arial" , sans-serif; font-size: 10.0pt;">Nelson Sousa</span></a></span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt;"> - 10 WTF moments in Pentaho
Data Integration<o:p></o:p></span></div>
<div class="MsoNormal" style="mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;">
<span lang="EN-US" style="mso-ansi-language: EN-US;"><br style="mso-special-character: line-break;" />
<!--[if !supportLineBreakNewLine]--><br style="mso-special-character: line-break;" />
<!--[endif]--></span><span lang="EN-US" style="font-family: "times new roman" , serif; font-size: 12.0pt; line-height: 107%;"><o:p></o:p></span></div>
<div align="center" class="MsoNormal" style="mso-element-anchor-horizontal: column; mso-element-top: .05pt; mso-element-wrap: auto; mso-element: frame; mso-height-rule: exactly; text-align: center;">
If you haven't done so, <a href="https://it-novum.com/en/pcm17/">Register Here</a></div>
<table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="mso-cellspacing: 0cm; mso-padding-alt: 0cm 0cm 0cm 0cm; mso-yfti-tbllook: 1184; width: 100%px;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0; mso-yfti-lastrow: yes;">
<td style="padding: 0cm 0cm 0cm 0cm;" valign="top"><table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="mso-cellspacing: 0cm; mso-padding-alt: 0cm 0cm 0cm 0cm; mso-yfti-tbllook: 1184; width: 100%px;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0; mso-yfti-lastrow: yes;">
<td style="padding: 0cm 0cm 0cm 0cm; width: 487.5pt;" valign="top" width="488"><div align="center" class="MsoNormal" style="mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly; text-align: center;">
<span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2872"><span lang="EN-US" style="background: #1b537d; border: solid #b5b5b5 1.0pt; color: white; display: none; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%; padding: 9.0pt;">Jetzt kostenlos
registrieren </span></a></span><span lang="EN-US" style="font-size: 10.0pt; line-height: 107%; mso-ansi-language: EN-US; mso-fareast-font-family: "Times New Roman";"><o:p></o:p></span></div>
</td>
</tr>
</tbody></table>
</td>
</tr>
</tbody></table>
<table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="mso-cellspacing: 0cm; mso-padding-alt: 0cm 0cm 0cm 0cm; mso-yfti-tbllook: 1184; width: 100%px;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0; mso-yfti-lastrow: yes;">
<td style="padding: 0cm 0cm 0cm 0cm;" valign="top"><table border="0" cellpadding="0" cellspacing="0" class="MsoNormalTable" style="mso-cellspacing: 0cm; mso-padding-alt: 0cm 0cm 0cm 0cm; mso-yfti-tbllook: 1184; width: 100%px;">
<tbody>
<tr style="mso-yfti-firstrow: yes; mso-yfti-irow: 0; mso-yfti-lastrow: yes;">
<td style="padding: 0cm 0cm 0cm 0cm;" valign="top"><div class="MsoNormal" style="mso-element-anchor-horizontal: column; mso-element-anchor-vertical: paragraph; mso-element-top: .05pt; mso-element-wrap: around; mso-element: frame; mso-height-rule: exactly;">
<span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%;"><br />
We are looking forward to seeing you in </span><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2886"><span lang="EN-US" style="color: #db0031; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%;">Mainz</span></a></span><span lang="EN-US" style="color: black; font-family: "arial" , sans-serif; font-size: 10.0pt; line-height: 107%;">, which can be
reached in only 20 minutes by train from Frankfurt airport or main train
station!<br />
</span><span lang="EN-US" style="mso-ansi-language: EN-US;">In the meantime follow-up on all updates
</span><span lang="DE"><a href="http://it-novum.acemlnd.com/lt.php?s=e0a0a3ec84fbc7dd9dad0cb6f9a41648&i=281A338A21A2876"><span lang="EN-US" style="color: #db0031; mso-ansi-language: EN-US;">on Twitter</span></a></span><span lang="EN-US" style="mso-ansi-language: EN-US;">. </span><span lang="EN-US" style="font-size: 12.0pt; line-height: 107%; mso-ansi-language: EN-US;"><o:p></o:p></span><br />
<span lang="EN-US" style="mso-ansi-language: EN-US;"><br /></span>
<span lang="EN-US" style="mso-ansi-language: EN-US;"><br /></span>
<span lang="EN-US" style="mso-ansi-language: EN-US;">-pedro, with all the content from this post shamelessly stolen from Ruth and Carolin, the spectacular organizers from <a href="https://it-novum.com/">IT-</a></span><a href="https://it-novum.com/">Novum</a></div>
</td></tr>
</tbody></table>
</td></tr>
</tbody></table>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com2tag:blogger.com,1999:blog-6644329693530300467.post-22882852211860351772017-08-14T10:59:00.002+01:002017-08-14T11:04:41.711+01:00Pentaho Maven repository changed to nexus.pentaho.orgFrom a <a href="https://github.com/pentaho/mondrian/issues/853">recent (at the time of writing, obviously!) issue</a> in the mondrian project we noticed we failed to notify an important change:<div>
<br /></div>
<blockquote class="tr_bq">
<div style="background-color: white; box-sizing: border-box; color: #24292e; font-family: -apple-system, system-ui, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 14px; margin-bottom: 16px;">
This morning the pentaho maven repository seems to be down.<br style="box-sizing: border-box;" />Each download request during maven build fails with 503 error:</div>
<div style="background-color: white; box-sizing: border-box; color: #24292e; font-family: -apple-system, system-ui, "Segoe UI", Helvetica, Arial, sans-serif, "Apple Color Emoji", "Segoe UI Emoji", "Segoe UI Symbol"; font-size: 14px;">
<code style="background-color: rgba(27, 31, 35, 0.05); border-radius: 3px; box-sizing: border-box; font-family: SFMono-Regular, Consolas, "Liberation Mono", Menlo, Courier, monospace; font-size: 11.9px; margin: 0px; padding: 0.2em 0px;">[WARNING] Could not transfer metadata XXX/maven-metadata.xml from/to pentaho-releases (http://repository.pentaho.org/artifactory/repo/): Failed to transfer file: http://repository.pentaho.org/artifactory/repo/XXX/maven-metadata.xml. Return code is: 503 , ReasonPhrase:Service Temporarily Unavailable.</code></div>
</blockquote>
<br />
The reason for this is that the maven url is now <i>nexus.pentaho.org/content/groups/omni</i> .<br />
<br />
Here's a link to a complete ~/.m2/settings.xml config file: <a href="https://github.com/pentaho/maven-parent-poms/blob/master/maven-support-files/settings.xml">https://github.com/pentaho/maven-parent-poms/blob/master/maven-support-files/settings.xml</a><br />
<br />
<br />
<br />
-pedroPedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com1tag:blogger.com,1999:blog-6644329693530300467.post-46129454013239945892017-07-26T21:21:00.002+01:002017-07-26T21:21:31.873+01:00PCM17 - Pentaho Community Meeting: November 10-12, Mainz<h2>
PCM17 - 10th Edition</h2>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://it-novum.com/fileadmin/user_upload/Hintergrundbilder/PCM17-Banner_EN.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="250" data-original-width="800" height="200" src="https://it-novum.com/fileadmin/user_upload/Hintergrundbilder/PCM17-Banner_EN.jpg" width="640" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
One of my favourite blog posts of the year - Announcing PCM17. And this year, for the 10th edition, we're going <a href="http://wiki.pentaho.com/display/COM/Pentaho+Community+Gathering+-+Mainz+2008">back to the beginning</a> - Mainz in Germany.</div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Location</h3>
<div>
Location address: Kupferbergterrasse, Kupferbergterrasse 17-19, 55116 Mainz. Close to Frankfurt, Germany</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRFXjNoLSxxy4ID5IIprWahQ9RwzEU9IRLUMaoFjSvGNkxhUjQfJopllbMA1Yi2rznICguQKA8Hs5yakmWH651O6D-ROXzH0N8e5R-DHQqn-fSpFTWVtgxPhOAdSx8yw5D0Bzj_5dE3OA/s1600/map.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="460" data-original-width="696" height="263" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiRFXjNoLSxxy4ID5IIprWahQ9RwzEU9IRLUMaoFjSvGNkxhUjQfJopllbMA1Yi2rznICguQKA8Hs5yakmWH651O6D-ROXzH0N8e5R-DHQqn-fSpFTWVtgxPhOAdSx8yw5D0Bzj_5dE3OA/s400/map.png" width="400" /></a></div>
<div>
<br /></div>
<h3>
Event</h3>
<div>
We're maintaining the schedule of the previous years: A meet-up on friday for drinks preceded by a hackathon; A meet-up on Saturday for drinks preceded by a bunch of presentations or really cool stuff; A meet-up on Sunday for drinks preceded by a city sightseeing! You got the idea</div>
<div>
<br /></div>
<h3>
All the information....</h3>
<a href="https://it-novum.com/en/pcm17/">Here: https://it-novum.com/en/pcm17/!</a> IT-Novum is doing a spectacular work organizing this event, and you'll find all the information needed, from instructions on how to get there to suggestions for hotels to stay on<br />
<br />
<h4>
Registration and Call for Presentations</h4>
<div>
Please go to the <a href="https://it-novum.com/en/pcm17/">#PCM17</a> website to register and also to send us a presentation proposal!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Cheers!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
-pedro</div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com2tag:blogger.com,1999:blog-6644329693530300467.post-51311531467461432982017-06-27T15:22:00.001+01:002017-06-27T15:22:50.186+01:00A consulting POV: Stop thinking about Data Warehouses!
<!--[if !mso]>
<style>
v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style>
<![endif]--><!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:AllowPNG/>
<o:PixelsPerInch>96</o:PixelsPerInch>
</o:OfficeDocumentSettings>
</xml><![endif]-->
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves>false</w:TrackMoves>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-GB</w:LidThemeOther>
<w:LidThemeAsian>X-NONE</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="false"
DefSemiHidden="false" DefQFormat="false" DefPriority="99"
LatentStyleCount="382">
<w:LsdException Locked="false" Priority="0" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="0" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 9"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="header"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footer"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index heading"/>
<w:LsdException Locked="false" Priority="35" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of figures"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope return"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="line number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="page number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of authorities"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="macro"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="toa heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 5"/>
<w:LsdException Locked="false" Priority="10" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Closing"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Signature"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Message Header"/>
<w:LsdException Locked="false" Priority="11" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Salutation"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Date"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Block Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="FollowedHyperlink"/>
<w:LsdException Locked="false" Priority="22" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Document Map"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Plain Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="E-mail Signature"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Top of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Bottom of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal (Web)"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Acronym"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Cite"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Code"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Definition"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Keyboard"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Preformatted"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Sample"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Typewriter"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Variable"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Table"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation subject"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="No List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Contemporary"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Elegant"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Professional"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Balloon Text"/>
<w:LsdException Locked="false" Priority="39" Name="Table Grid"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Theme"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 9"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" QFormat="true"
Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" QFormat="true"
Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" QFormat="true"
Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" QFormat="true"
Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" QFormat="true"
Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" QFormat="true"
Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" SemiHidden="true"
UnhideWhenUsed="true" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="TOC Heading"/>
<w:LsdException Locked="false" Priority="41" Name="Plain Table 1"/>
<w:LsdException Locked="false" Priority="42" Name="Plain Table 2"/>
<w:LsdException Locked="false" Priority="43" Name="Plain Table 3"/>
<w:LsdException Locked="false" Priority="44" Name="Plain Table 4"/>
<w:LsdException Locked="false" Priority="45" Name="Plain Table 5"/>
<w:LsdException Locked="false" Priority="40" Name="Grid Table Light"/>
<w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="46" Name="List Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Mention"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Smart Hyperlink"/>
</w:LatentStyles>
</xml><![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Courier New";
panose-1:2 7 3 9 2 2 5 2 4 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:fixed;
mso-font-signature:-536859905 -1073711037 9 0 511 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;
mso-font-charset:2;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:0 268435456 0 0 -2147483648 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:roman;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:swiss;
mso-font-pitch:variable;
mso-font-signature:-536870145 1073786111 1 0 415 0;}
@font-face
{font-family:"Open Sans";
mso-font-alt:Arial;
mso-font-charset:0;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:-536870161 1073750107 40 0 415 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-parent:"";
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:7.2pt;
margin-left:0cm;
line-height:120%;
mso-pagination:widow-orphan;
font-size:12.0pt;
mso-bidi-font-size:11.0pt;
font-family:"Open Sans";
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
h1
{mso-style-priority:9;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"Heading 2";
mso-style-link:"Heading 1 Char";
mso-style-next:Normal;
margin-top:0cm;
margin-right:0cm;
margin-bottom:18.0pt;
margin-left:0cm;
line-height:107%;
mso-pagination:widow-orphan;
page-break-after:avoid;
mso-outline-level:1;
font-size:28.0pt;
font-family:"Open Sans";
mso-bidi-font-family:"Open Sans";
color:#005DA6;
mso-font-kerning:0pt;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;
font-weight:normal;}
h2
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-link:"Heading 2 Char";
mso-style-next:Normal;
margin-top:24.0pt;
margin-right:0cm;
margin-bottom:18.0pt;
margin-left:0cm;
line-height:107%;
mso-pagination:widow-orphan;
page-break-after:avoid;
mso-outline-level:2;
font-size:18.0pt;
mso-bidi-font-size:16.0pt;
font-family:"Open Sans";
mso-bidi-font-family:"Open Sans";
color:#005DA6;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;
font-weight:normal;}
h3
{mso-style-priority:9;
mso-style-qformat:yes;
mso-style-link:"Heading 3 Char";
mso-style-next:Normal;
margin-top:24.0pt;
margin-right:0cm;
margin-bottom:14.4pt;
margin-left:0cm;
line-height:120%;
mso-pagination:widow-orphan;
page-break-after:avoid;
mso-outline-level:3;
font-size:14.0pt;
font-family:"Open Sans";
mso-bidi-font-family:"Open Sans";
color:gray;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;
font-weight:normal;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:7.2pt;
margin-left:36.0pt;
mso-add-space:auto;
line-height:120%;
mso-pagination:widow-orphan;
font-size:10.0pt;
mso-bidi-font-size:11.0pt;
font-family:"Open Sans";
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
color:#404040;
mso-themecolor:text1;
mso-themetint:191;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:6.0pt;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:120%;
mso-pagination:widow-orphan;
font-size:10.0pt;
mso-bidi-font-size:11.0pt;
font-family:"Open Sans";
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
color:#404040;
mso-themecolor:text1;
mso-themetint:191;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:120%;
mso-pagination:widow-orphan;
font-size:10.0pt;
mso-bidi-font-size:11.0pt;
font-family:"Open Sans";
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
color:#404040;
mso-themecolor:text1;
mso-themetint:191;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:7.2pt;
margin-left:36.0pt;
mso-add-space:auto;
line-height:120%;
mso-pagination:widow-orphan;
font-size:10.0pt;
mso-bidi-font-size:11.0pt;
font-family:"Open Sans";
mso-fareast-font-family:"Times New Roman";
mso-fareast-theme-font:minor-fareast;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
color:#404040;
mso-themecolor:text1;
mso-themetint:191;
mso-ansi-language:EN-US;
mso-fareast-language:EN-US;}
span.Heading1Char
{mso-style-name:"Heading 1 Char";
mso-style-priority:9;
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:"Heading 1";
mso-ansi-font-size:28.0pt;
mso-bidi-font-size:28.0pt;
font-family:"Open Sans";
mso-ascii-font-family:"Open Sans";
mso-hansi-font-family:"Open Sans";
mso-bidi-font-family:"Open Sans";
color:#005DA6;
mso-ansi-language:EN-US;}
span.Heading2Char
{mso-style-name:"Heading 2 Char";
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:"Heading 2";
mso-ansi-font-size:18.0pt;
mso-bidi-font-size:16.0pt;
font-family:"Open Sans";
mso-ascii-font-family:"Open Sans";
mso-hansi-font-family:"Open Sans";
mso-bidi-font-family:"Open Sans";
color:#005DA6;
mso-ansi-language:EN-US;}
span.Heading3Char
{mso-style-name:"Heading 3 Char";
mso-style-priority:9;
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:"Heading 3";
mso-ansi-font-size:14.0pt;
mso-bidi-font-size:14.0pt;
font-family:"Open Sans";
mso-ascii-font-family:"Open Sans";
mso-hansi-font-family:"Open Sans";
mso-bidi-font-family:"Open Sans";
color:gray;
mso-ansi-language:EN-US;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-fareast-language:EN-US;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;
mso-header-margin:35.4pt;
mso-footer-margin:35.4pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:731319850;
mso-list-type:hybrid;
mso-list-template-ids:1408816598 134807567 134807577 134807579 134807567 134807577 134807579 134807567 134807577 134807579;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1
{mso-list-id:1376082426;
mso-list-type:hybrid;
mso-list-template-ids:-1402049084 134807567 134807577 134807579 134807567 134807577 134807579 134807567 134807577 134807579;}
@list l1:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l1:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l2
{mso-list-id:1990791260;
mso-list-type:hybrid;
mso-list-template-ids:1309985466 1375656010 134807555 134807557 134807553 134807555 134807557 134807553 134807555 134807557;}
@list l2:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
@list l2:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New",serif;}
@list l2:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New",serif;}
@list l2:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l2:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l2:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New",serif;}
@list l2:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
-->
</style>
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0cm 5.4pt 0cm 5.4pt;
mso-para-margin:0cm;
mso-para-margin-bottom:.0001pt;
mso-pagination:widow-orphan;
font-size:12.0pt;
font-family:"Calibri",sans-serif;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-fareast-language:EN-US;}
</style>
<![endif]-->
<!--StartFragment-->
<h1>
<span lang="EN-US">A consulting POV: Stop thinking about Data Warehouses!<o:p></o:p></span></h1>
<div class="MsoNormal">
<i><span lang="EN-US">What
I am writing in here is the materialization of a line of thought that started
bothering me a couple of years ago. While I implemented projects after projects,
built ETLs, optimized reports, designed dashboards, I couldn’t help but
thinking that something didn’t quite make sense, but couldn’t quite see what.
When I tried to explain it to someone, I just got blank stares… <o:p></o:p></span></i></div>
<div class="MsoNormal">
<i><span lang="EN-US">Eventually
things started to make more sense to me (which is far from saying they actually
make sense, as I’m fully aware my brain is, hum, let’s just say a little bit
messed up!) and I ended up realizing that I’ve been looking at the challenges
from a wrong perspective. And while this may seem a very small change in
mindset (specially if I fail in passing the message, which may very well
happen), the implications are huge: not only it changed our methodology on how
to implement projects in our services teams, it’s also guiding Pentaho’s
product development and vision.<o:p></o:p></span></i></div>
<div class="MsoNormal">
<i><span lang="EN-US"><br /></span></i></div>
<h2>
<span lang="EN-US">A few years ago, in a blog post far, far away...<o:p></o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">A couple of years ago I wrote a blog post
called <i>”Kimball is getting old”</i>. It focused on one fundamental point:
technology was evolving to a point where just looking at the concept of an
enterprise datawarehouse (EDW) seemed restrictive. After all, the end users
care only about information; they couldn’t care less about what gets the
numbers in front of them. So I proposed that we should apply a very critical
eye to our problem, and maybe, sometimes, Kimball’s DW, with its star schemas,
snowflakes and all that jazz wasn’t the best option and we should choose
something else…<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<h2>
<span lang="EN-US">But I wasn’t completely right…<o:p></o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">I’m still (more than ever?) a huge
proponent of the <i>top down</i> approach: focus on usability, focus on the
needs of the user, provide him a great experience. All rest follows. All of
that is still spot on.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">But I made 2 big mistakes:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">I confused data modelling with
data warehouse<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">I kept seeing data sources
conceptually as the unified, monolithic source of every insight<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<br /></div>
<h2>
<span lang="EN-US">Data Modelling – the semantics behind the data<o:p></o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">Kimball was a bloody genius! Actually, my
mistake here was actually due to the fact that he is <i>way</i> smarter than
everyone else. Why do I say this? Because he didn’t come up with one, but with <i>two</i> groundbreaking ideas... <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">First, he realized that the value of data,
business-wise, comes when we stop considering it as just zeros and ones and
start treating it as business concepts. That’s what the <i>Data Modelling</i>
does: By adding semantics to raw data, immediately gives it meaning that makes
sense to a wide audience of people. And this is the part that I erroneously
dismissed. This is still spot on! All his concepts of dimensions, hierarchies,
levels and attributes, are relevant first and foremost because that’s how
people <i>think</i>. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">And then, he immediately went prescriptive
and told us <i>how</i> we could map those concepts to database tables and answer
the business questions with relational database technology with concepts like
star schemas, snowflake, different types of slowly changing dimensions, aggregation
techniques, etc.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">He did such a good job that he basically shaped
how we worked; How many of us were involved in projects where we were talked to
build data warehouses to give all possible answers when we didn’t even know the
questions? I’m betting a lot, I certainly did that. We were taught to provide
answers without focusing on understanding the questions.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<h2>
<span lang="EN-US">Project’s complexity is growing exponentially<o:p></o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">Classically, a project implementation was
simply around reporting on the past. We can’t do that anymore; If we want our
project to succeed, it can’t just report on the past: It also has to describe
the present and predict the future.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">There’s also the explosion on the amount of
data available.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">IoT brought us an entire new set of devices
that are generating data we can collect.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Social media and behavior analysis brought
us closer to our users and customers<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">In order to be impactful (regardless of how
“impact” is defined), a BI project <i>has</i> to trigger operational actions:
schedule maintenances, trigger alerts, prevent failures. So, bring on all those
data scientists with their predictive and machine learning algorithms... <i><o:p></o:p></i></span></div>
<div class="MsoNormal">
<span lang="EN-US">On top of that, in the past, we might have
been successful at convincing our users that it’s perfectly reasonable to expect
a couple of hours for that monthly sales report that processed a couple of
gigabytes of data. We all know that’s changed; if they can search the entire
internet in less than a second, why would they waste <i>minutes</i> for a “small”
report?? And let’s face it, they’re right…<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">The consequence? It’s getting much more
complex to define, architect, implement, manage and support a project that
needs more data, more people, more tools.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Am I making all of this sound like a bad
thing? On the contrary! This is a great problem to have! In the past, BI
systems were confined to delivering analytics. We’re now given the chance to
have a much bigger impact in the world! Figuring this out is actually the only
way forward for companies like Pentaho: We either succeed and grow, or we
become irrelevant. And I certainly don’t want to become irrelevant!<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<h2>
<span lang="EN-US">IT’s version of the Heisenberg’s Uncertainty Principle: Improving
both speed <i>and</i> scalability??<o:p></o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">So how do we do this?<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">My degree is actually in Physics (don’t
pity me, took me a while but I eventually moved away from that), and even
though I’m a really crappy one, I do know some of the basics…<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">One of the most well-known theorems in
physics is Heisenberg’s Uncertainty principle. You cannot accurately know both
the speed and location of (sub-)atomic particle with full precision. But can
have a precise knowledge over one in detriment of the other<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">I’m very aware this analogy is a little bit
silly (to say the least) but it’s at least vivid enough on my mind to make me
realize that we can’t expect in IT to solve both the speed and scalability
issue – at least not to a point where we have a one size fits all approach. <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">There have been spectacular improvements in
the distributed computing technologies – but all of them have their pros and
cons, the days where a database was good for all use cases is long gone.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">So what do we do for a project where we
effectively need to process a bunch of data and at the same time it has to be
blazing fast? What technology do we chose?<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<h2>
<span lang="EN-US">Thinking “data sources” slightly differently<o:p></o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">When we think about data sources, there are
2 traps most of us fall into:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US">1.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">We think of them as a monolithic
entity (eg: <i>Sales</i>, <i>Human
Resources</i>, etc) that hold all the information relevant to a topic<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US">2.<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">We think of them from a
technology perspective <o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Let me try to explain this through an example.
Imagine the following customer requirement, here in the format of a dashboard,
but could very well be any other delivery format (yeah, cause a dashboard, a
report, a chart, whatever, is just <i>the way</i> we chose to deliver the
information):<o:p></o:p></span></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<!--[if gte vml 1]><v:shapetype
id="_x0000_t75" coordsize="21600,21600" o:spt="75" o:preferrelative="t"
path="m@4@5l@4@11@9@11@9@5xe" filled="f" stroked="f">
<v:stroke joinstyle="miter"/>
<v:formulas>
<v:f eqn="if lineDrawn pixelLineWidth 0"/>
<v:f eqn="sum @0 1 0"/>
<v:f eqn="sum 0 0 @1"/>
<v:f eqn="prod @2 1 2"/>
<v:f eqn="prod @3 21600 pixelWidth"/>
<v:f eqn="prod @3 21600 pixelHeight"/>
<v:f eqn="sum @0 0 1"/>
<v:f eqn="prod @6 1 2"/>
<v:f eqn="prod @7 21600 pixelWidth"/>
<v:f eqn="sum @8 21600 0"/>
<v:f eqn="prod @7 21600 pixelHeight"/>
<v:f eqn="sum @10 21600 0"/>
</v:formulas>
<v:path o:extrusionok="f" gradientshapeok="t" o:connecttype="rect"/>
<o:lock v:ext="edit" aspectratio="t"/>
</v:shapetype><v:shape id="Picture_x0020_1" o:spid="_x0000_i1027" type="#_x0000_t75"
alt="../../../../Bamboo%20Paper/Pentaho%208%20-%20Page%203-S.png" style='width:202pt;
height:269pt;visibility:visible;mso-wrap-style:square'>
<v:imagedata src="file:////Users/palves/Library/Group%20Containers/UBF8T346G9.Office/msoclip1/01/clip_image001.png"
o:title="../../../../Bamboo%20Paper/Pentaho%208%20-%20Page%203-S.png"/>
</v:shape><![endif]--><!--[if !vml]--><!--[endif]--><span lang="EN-US"><o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZq4LCMstF7Vmmm8d1dOilgA4hqZ7sGlU643tcxVbBtXoK_sDP5BZEbrpd8PUDoaAqmrNNBuQpO6elQrGg0L1lAEI-GzJj8H_gPoTaYc62O1dG5g2QVHGLL3ZK0BxZD1Di67c7E5K9Mr4/s1600/Pentaho+8+-+Page+3-S.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="800" data-original-width="600" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjZq4LCMstF7Vmmm8d1dOilgA4hqZ7sGlU643tcxVbBtXoK_sDP5BZEbrpd8PUDoaAqmrNNBuQpO6elQrGg0L1lAEI-GzJj8H_gPoTaYc62O1dG5g2QVHGLL3ZK0BxZD1Di67c7E5K9Mr4/s400/Pentaho+8+-+Page+3-S.png" width="300" /></a></div>
<div class="MsoNormal">
<span lang="EN-US"><br /></span></div>
<div class="MsoNormal">
<span lang="EN-US">Pretty common, hum?<o:p></o:p></span></div>
<h3>
<span lang="EN-US">The classical approach<o:p></o:p></span></h3>
<div class="MsoNormal">
<span lang="EN-US">When thinking about this (common) scenario from
the classical implementation perspective, the first instinct would be to start designing
a data warehouse (doesn’t even need to be an EDW <i>per se</i>, could be Hadoop, a no-sql source, etc).
We would build our ETL process (with PDI or whatever) from the source systems
through an ETL and there would always be a stage of modelling so we could get
to our <i>Sales</i> data source that could answer
all kinds of questions.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">After that is done, we’d be able to write
the necessary queries to generate the numbers our fictitious customer wants.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">And after a while, we would implement a
solution architecture diagram similar to this, that I’m sure looks very similar
to everything we’ve all been doing in consulting:<o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3DWbR2cgfZMCA5d8lKlFmDtqj3ztM_8RkB27ZYyEJowskSG0iBJ5C8NE6TKUZ8QTFfhi8wQK8kCQtYd7bEVildR651bUT4WPn9tRA0SbxJAuE4WQXxAJKdnp152e8_7zraU5hJ0CAXSY/s1600/Pentaho+8+-+Page+4-S.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="600" data-original-width="800" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3DWbR2cgfZMCA5d8lKlFmDtqj3ztM_8RkB27ZYyEJowskSG0iBJ5C8NE6TKUZ8QTFfhi8wQK8kCQtYd7bEVildR651bUT4WPn9tRA0SbxJAuE4WQXxAJKdnp152e8_7zraU5hJ0CAXSY/s400/Pentaho+8+-+Page+4-S.png" width="400" /></a></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US">Our customer gets the number he numbers he
want, he’s happy and successful. So successful that he expands, does a bunch of
acquisitions, gets so much data that our system starts to become slow. The
sales “table” never stops growing. It’s a pain to do anything with it… Part of
our dashboard takes a while to render… we’re able to optimize part of it, but
other areas become slow.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">In order to optimize the performance and
allow the system to scale, we consider changing the technology. From relational
databases to vertical column store databases, to nosql data stores, all the way
through Hadoop, in a permanent effort to keep things scaling and fast…<o:p></o:p></span></div>
<h3>
<span lang="EN-US">The business’ approach<o:p></o:p></span></h3>
<div class="MsoNormal">
<span lang="EN-US">Let’s take a step back. Looking at our
requirements, the main KPI the customer wants to know is: <o:p></o:p></span></div>
<div class="MsoNormal" style="text-indent: 36.0pt;">
<i><span lang="EN-US">How much did
I sell yesterday and how is that compared to budget?<o:p></o:p></span></i></div>
<div class="MsoNormal">
<span lang="EN-US">It’s <i>one
number</i> he’s interested in.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Look at
the other elements: He wants the top reps for the month. He wants a chart for
the MTD sales. How many data points is that? 30 tops? I’m being simplistic on
purpose, but the thing is that it is extremely stupid to force ourselves to
always go through all the data when the vast majority of the questions isn’t a <i>big
data</i> challenge in the first place. It may <i>need</i> big data processing
and orchestration, but certainly not at runtime.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">So here’s
how I’d address this challenge<o:p></o:p></span></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzrkUWt4cHTLh194dpnn0nvoxkbrwjcF6KlGtHlLQ9Wg6KFkZkDcs-3J7AY7X03PdMHCb4D7X8PFUrWwobHCXP1OriWrSWz7OivNV8DfKEkUakoSDH7IylJdOt76O-JhMDq-6lnQSsLiQ/s1600/Pentaho+8+-+Page+5-S.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" data-original-height="600" data-original-width="800" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjzrkUWt4cHTLh194dpnn0nvoxkbrwjcF6KlGtHlLQ9Wg6KFkZkDcs-3J7AY7X03PdMHCb4D7X8PFUrWwobHCXP1OriWrSWz7OivNV8DfKEkUakoSDH7IylJdOt76O-JhMDq-6lnQSsLiQ/s400/Pentaho+8+-+Page+5-S.png" width="400" /></a></div>
<div align="center" class="MsoNormal" style="text-align: center;">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US">I would
focus on the business question. I would not do a single <i>Sales</i>
datasource. Instead, I’d define the following <i>Business Data Sources</i>
(sorry, I’m not very good at naming stuff..), and I’d force myself to define them
in a way where each of them contains (or output) a small set of data (up to a
few millions the most):<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">ActualVsBudgetThisMonth<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">CustomerSatByDayAndStore<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">SalesByStore<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">SalesRepsPerformance<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Then I’d implement these <i>however I needed</i>!
Materialized, unmaterialized, database or Hadoop, whatever worked. But through
this exercise we define a clear separation between where all the data is and
the most common questions we need to answer in a very fast way.<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Does something like this gives us all the
liberty to answer all the questions? Absolutely not! But at least for me doesn’t
make a lot of sense to optimize a solution to give answers when I don’t even
know what the questions are. And the big data store is still there somewhere
for the data scientists to play with<o:p></o:p></span></div>
<div class="MsoNormal">
<span lang="EN-US">Like I said, while the differences may seem
very subtle at first, here are some advantages I found of thinking through
solution architecture this way:<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">Faster to implement – since our
<i>business datasources</i>’s <i>signature</i> is much smaller and well
identified, it’s much easier to fill in the blanks<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">Easier to validate – since the
datasources are smaller, they are easier to validate with the business
stakeholders as we lock them down and move to other business data sources<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">Technology agnostic – note that
at any point in time I mentioned technology choices. Think of these datasources
as an API<o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<!--[if !supportLists]--><span lang="EN-US" style="font-family: Symbol; mso-bidi-font-family: Symbol; mso-fareast-font-family: Symbol;">·<span style="font-family: "Times New Roman"; font-size: 7pt; font-stretch: normal; font-variant-numeric: normal; line-height: normal;">
</span></span><!--[endif]--><span lang="EN-US">Easier to optimize – since we
split a big data sources in multiple smaller ones, they become easier to
maintain, support and optimize <o:p></o:p></span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l2 level1 lfo3; text-indent: -18.0pt;">
<span lang="EN-US"><br /></span></div>
<h2>
<span lang="EN-US">Concluding thoughts<o:p></o:p></span></h2>
<div class="MsoNormal">
<span lang="EN-US">Give it a try – this will seem odd at
first, but it forces us to think differently. We spend too much time worrying
about the technology that more than often we forget what we’re here to do in
the first place…<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<span lang="EN-US">-pedro<o:p></o:p></span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<br /></div>
<!--EndFragment-->Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com4tag:blogger.com,1999:blog-6644329693530300467.post-90343147546242122772017-05-22T20:09:00.001+01:002017-05-24T07:43:42.895+01:00Pentaho 7.1 is available! <h2>
Pentaho 7.1 is out</h2>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3M7sKcVm5GU1ooGzNrPJCgrvZ-pyI_a0lA45HCVSTmXl72-p0INGsHDrPu-Mv2AEX5SNe8zBeqR_oDqKW47Jl02JWU9FPdQ1-Z_uAI1IgzE-pUeWSvwKP46kSYAr6GAWiwlP95bQauzU/s1600/pentaho7.1.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="206" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi3M7sKcVm5GU1ooGzNrPJCgrvZ-pyI_a0lA45HCVSTmXl72-p0INGsHDrPu-Mv2AEX5SNe8zBeqR_oDqKW47Jl02JWU9FPdQ1-Z_uAI1IgzE-pUeWSvwKP46kSYAr6GAWiwlP95bQauzU/s400/pentaho7.1.png" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
Remember when I said at the time of the <a href="http://pedroalves-bi.blogspot.pt/2016/10/pentaho-7.0.html">previous release</a> that Pentaho 7.0 was the best release ever? Well, I was true till today! But not any more, as <a href="http://www.pentaho.com/product/version-7-1-update">7.1 is even better</a>! :p</div>
<div>
<br /></div>
<div>
Why do I say that? It's a big step forward in the direction we've been aiming - consolidating and simplifying our stack, not passing the complexity to the end user. </div>
<div>
<br /></div>
<div>
These are the main features in the release:</div>
<div>
<br /></div>
<div>
<ul>
<li>Visual Data Experience</li>
<ul>
<li>Data Exploration (PDI)</li>
<ul>
<li>Drill Down</li>
<li>New Viz's: Geo map, sunburst, Heat Grid</li>
<li>Tab Persistency</li>
<li>Several other improvements including performance</li>
</ul>
<li>Viz API 3.0 (Beta)</li>
<ul>
<li>Viz API 3.0, with documentatino</li>
<li>Rollout of consistent visualizations between Analyzer, PDI and Ctools</li>
</ul>
</ul>
<li>Enterprise Platform</li>
<ul>
<li>VCS-friendly features</li>
<ul>
<li>File / repository abstraction</li>
<li>PDI files properly indented</li>
<li>Repository performance improvements</li>
</ul>
<li>Reintroducing Ops Mart</li>
<li>New default theme on User Console</li>
<li>Pentaho Mobile deprecation</li>
</ul>
<li>Big Data Innovation</li>
<ul>
<li>AEL - Adaptive Execution Layer (via Spark)</li>
<li>Hadoop Security</li>
<ul>
<li>Kerberos Impersonation (for Hortonworks)</li>
<li>Ranger support</li>
</ul>
<li>Microsoft Azure HD Insights shim</li>
</ul>
</ul>
</div>
<div>
<br /></div>
<div>
I'm getting tired just of listing all this stuff... Now into a bit more detail, and I'll jump back and forth in these different topics ordering by the ones that... well, that I like the most :p<br />
<br /></div>
<h3>
Adaptive Execution with Spark</h3>
<div>
This is huge; We've decoupled the execution engine from PDI so we can plug in other engines. Now we have 2: </div>
<div>
<ul>
<li>Pentaho - the classic pentaho engine</li>
<li>Spark - you've guessed it...</li>
</ul>
<div>
What's the goal of this? Making sure we treat our ETL development with a <i>pay as you go</i> approach; First, we worry about the logic, then we select the engine that makes most sense.</div>
</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="http://www.pentaho.com/sites/default/files/uploads/images/adaptiveexecution-v1.gif" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="276" src="https://www.pentaho.com/sites/default/files/uploads/images/adaptiveexecution-v1.gif" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>AEL execution of Spark</i></td></tr>
</tbody></table>
<div>
One of the things people need to do on other tools (and even on our own tools, that's why I don't like our own approach to the Pentaho Map Reduce) is that from the start you need to think about the engine and technology you're going to use. But this makes little sense.<br />
<br />
<h4>
Scale as you go</h4>
<h4>
<span style="font-weight: normal;">Pentaho’s message is one of future-proofing the IT architecture, leveraging the best of what the different technologies have to offer without imposing a certain configuration or persona as the starting point. The market is moving towards a demand for BA/DI to come together in a single platform. Pentaho has an advantage here as we have seen the differentiation of BI and DI better together with our customers and what sets us apart from the competition. Gartner predicts that BI and Discovery tool vendors will partner to accomplish this. Larger, proprietary vendors, will attempt to build these platforms themselves. With this approach from the competition, Pentaho has a unique and early lead in delivering this platform.</span></h4>
<h4>
<span style="font-weight: normal;">A good example is the story we can tell about governed blending. We don’t need to impose on customers any pre-determined configuration; We can start with the simple use of dataservices and unmaterialized data sets. If it’s fast enough, we’re done. If not, we can materialize the data into a data base or even an enterprise data warehouse. If it’s fast enough, we’re done. If not we can resort to other technologies – NoSQL, Lucene based engines, etc. If it’s fast enough, we’re done. If everything else fails, we can setup a SDR blueprint which is the ultimate scalability solution. And throughout this entire journey we never let go of the governed blending message.</span></h4>
<h4>
<span style="font-weight: normal;">This is an insanely powerful and differentiated message; We allow our customers to start simple, and only go down the more complex routes when needed. When going down a single path, a user knows, accepts and sees the value in extra complexity to address scalability </span></h4>
</div>
<h4>
Adaptive Execution Layer</h4>
<div>
<div>
The strategy described for the “Logical Data Warehouse” is exactly the one we need for the execution environment; A lot of times customers get hung up on a certain technology without even understanding if they actually needed. Countless times we we’ve seen customers asking for Spark without a use case that justifies it. We have to challenge that.</div>
<div>
<br /></div>
<div>
We need to move towards a scenario where the customer doesn’t have to think about technology first. We’ll offer one single approach and ways to scale as needed. If a data integration job works on a single Pentaho Server, why bother with other stacks? if it’s not enough, then making the jump to something like Map Reduce or Spark has to be a linear move.</div>
<div>
<br /></div>
<div>
The following diagram shows the Adaptive Execution Layer approach just described</div>
</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzxcIwEBh06Fxl_6CNUJAYhbhputD3jUpKggofMikYu-4jvqBZZ5X7sRF9bEC-K5KzOpQP2E7Bd98vNGhI1MtvBA0u_dMeNTNpMbiu9n1sceVSu8clTNszjmB5l4_aVMp2UzWK6PXujbY/s1600/AEL.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="290" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgzxcIwEBh06Fxl_6CNUJAYhbhputD3jUpKggofMikYu-4jvqBZZ5X7sRF9bEC-K5KzOpQP2E7Bd98vNGhI1MtvBA0u_dMeNTNpMbiu9n1sceVSu8clTNszjmB5l4_aVMp2UzWK6PXujbY/s400/AEL.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>AEL conceptual diagram</i></td></tr>
</tbody></table>
<div>
<br /></div>
<h4>
Implementation in 7.1 - Spark</h4>
<div>
<div>
For 7.1 we chose Spark as the first engine to implement for AEL. It has seen a lot of adoption, and the fact that it's not restricted to a map reduce paradigm makes it a good candidate to separate business logic and execution.</div>
<div>
<br /></div>
<div>
How to make it work? This high definition conceptual diagram should help me explain it:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDCF7i0EKKO5wK7xq-JIbyCTlq2YVhZz2JyWp3-GzEpBjAUupI8_Of8Tjl-ziqQPnKAwxvdf_7uuAgTia1GnvX9NenUGDNrfUGO0xM1lh8xlj_2GttoSzhw6wQJF6mIkszBAXJ_Wv-Lpo/s1600/ael-sketch.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="300" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgDCF7i0EKKO5wK7xq-JIbyCTlq2YVhZz2JyWp3-GzEpBjAUupI8_Of8Tjl-ziqQPnKAwxvdf_7uuAgTia1GnvX9NenUGDNrfUGO0xM1lh8xlj_2GttoSzhw6wQJF6mIkszBAXJ_Wv-Lpo/s400/ael-sketch.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>An architectural diagram so beautiful it should almost be roughly correct</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
We start by generating a PDI Driver for Spark from our own PDI instance. This is a very important starting point because using this methodology we ensure that any plugins we may have developed / installed will work when we run the transformation - we couldn't let go of the extensibility capabilities of Pentaho </div>
<div>
<br /></div>
<div>
That driver will be installed on an <i>edge node</i> of the cluster, and that's what will be responsible for executing the transformation. Note that by using spark we're leveraging all it's characteristics: namely, we don't even need a cluster, as we can select if we want to use spark standalone or yarn mode, even though I suspect the majority of users will be on yarn mode leveraging the clustering capabilities.</div>
<div>
<br /></div>
<div>
<h4>
Runtime flow</h4>
<div>
One of the main capabilities of AEL is that we don't need to think about adapting the business logic to the engine; We develop the transformation <i>first</i> and <i>then</i> we select where we want to execute. This is how this will work from within Spoon:</div>
</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEht14jU3sU6k8AEZl8IeiuVrn04PPa1Q2Dyz7ARO51bJSmAujebKteWQYxLJGRzPx_nB4ESSw5TLI-vbK3cJCLOCM3gIWNZYthbScOkD4Gc1eQ9MvISpYdu0ATrFTz9Mglq1JamwrK1VR8/s1600/RunOptions.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="298" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEht14jU3sU6k8AEZl8IeiuVrn04PPa1Q2Dyz7ARO51bJSmAujebKteWQYxLJGRzPx_nB4ESSw5TLI-vbK3cJCLOCM3gIWNZYthbScOkD4Gc1eQ9MvISpYdu0ATrFTz9Mglq1JamwrK1VR8/s400/RunOptions.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Creating and selecting a Spark run configuration</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
We created the concept of a <i><a href="https://help.pentaho.com/Documentation/7.1/0L0/0Y0/030/030/010#Run_Configurations">Run Configuration</a></i>. Once we select a run configuration set up to use Spark as the engine, PDI will send the transformation to the edge node and the driver will then execute it.</div>
<div>
<br /></div>
<div>
<i>All</i> transformation steps in PDI will run in AEL-Spark! This was the thought from the start. And to understand how this works, there are 2 fundamental concepts to understand:</div>
<div>
<br /></div>
<div>
<ul>
<li>Some steps are <i>safe to run in parallel</i> while others <i>are not parallelizable</i> or not recommended to run in clustered engines such as Spark. All the steps that take one row as input and one row as output (calculator, filter, select values, etc, etc), all of them are parallelizable; Steps that require access to other rows or depend on the position and order on the row set, still run on spark, but have to run on the edge node, which implies a collect of the RDDs (spark's datasets) from the nodes. It is what it is. And how do we know that? We simply tell PDI which steps are safe to run in parallel, and which are not</li>
<li>Some steps can leverage Spark's native APIs for perfomance and optimization. When that's the case, we can pass to PDI a <i>native implementation</i> of the step, greatly increasing the scalability on possible bottleneck points. Examples of these steps are the hadoop file inputs, hbase lookups, and many more</li>
</ul>
</div>
<br />
<h4>
Feedback please!</h4>
</div>
<div>
Even though running on secured clusters (and leveraging impersonation) is an EE capability only, AEL is also available in CE. Reason for that is that we want to get help from the community in testing, hardening, nativizing more steps and even writing more engines for AEL. So go and kick the tires of this thing! (and I'll surely do a blog post on this alone)</div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Visual Data Experience (PDI) Improvements</h3>
<div>
This is one of my favorite projects. You may be wondering what's the real value of having this improved data experience in PDI, why is this all that exciting... Let me tell you why: This is the first materialization of something that we hope becomes <i>the way</i> to handle data in pentaho <i>regardless of where we are</i>. So this thing that we're building in PDI, will eventually make it's way to the server... I'd like to throw away all the technicalities that we expose in our server (analyzer for olap, pir for metadata, prd for dashboards....) into a single content driver approach and usability experience. This is surely starting to sound confusing, so I better stop here :p</div>
<div>
<br /></div>
In the 7.1 release, Pentaho provides new Data Explorer capabilities to further support the following key use cases more completely:<br />
<ul>
<li><i>Data Inspection: </i>During the process of cleansing, preparing, and onboarding data, organizations often need to validate the quality and consistency of data across sources. Data Explorer enables easier identification of these issues, informing how PDI transformations can be adjusted to deliver clean data. </li>
<li><i>BI Prototyping:</i> As customers deliver analytic ready data to business analysts, Data Explorer reduces the iterations between business and IT. Specifically, It enables the validation of metadata models that are required for using Pentaho BA. Models can be created in PDI and tested in Data Explorer, ensuring data sources are analytics-ready when published to BA.</li>
</ul>
<div>
<br /></div>
<div>
And how? By adding these improvements:</div>
<h4>
New visualization: Heatgrid</h4>
This chart can display 2 measures (metrics) and 2 attributes (categories) at once. Attributes are displayed on the axes and measures are represented by the size and color of the points on the grid. It is most useful for comparing metrics at the ‘intersection’ of 2 dimensions, as seen in the comparisons of quantity and price across combinations of different territories and years below (did I just define what an heatgrid is?! No wonder it's taking me hours to write this post!):<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghQIlT1ibcEVxbWC8RJRC-8XQDRxPdkFhWuq_FjX6t40GoPXk9ayQJ35zRXv80ak62DHtaOTdY2JcmUbkVYh3bzE5LRwTAYrCu_ITj9EWRzEtaqItEvk2GzWx1ohGg0MnJh-W-m4R2Ko8/s1600/heatgrid.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="257" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghQIlT1ibcEVxbWC8RJRC-8XQDRxPdkFhWuq_FjX6t40GoPXk9ayQJ35zRXv80ak62DHtaOTdY2JcmUbkVYh3bzE5LRwTAYrCu_ITj9EWRzEtaqItEvk2GzWx1ohGg0MnJh-W-m4R2Ko8/s400/heatgrid.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Look at all those squares!</i></td></tr>
</tbody></table>
<div>
<span style="font-family: "calibri"; font-size: 12pt; line-height: 115%;"><br /></span></div>
<div>
<span style="line-height: 115%;"></span><br />
<h4 style="font-family: Times; font-size: medium;">
<span style="line-height: 115%;">
New visualization: Sunburst</span></h4>
<span style="line-height: 115%;">
</span>
<div>
<span style="line-height: 115%;">A pie chart on steroids that can show hierarchies. Less useless than a normal piechart!</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="font-family: Times; font-size: medium; margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg18PP8gw394vBbwUbPZN0MxA0uZ3NNWiFtsrpN668rBaQzN5IlOKsqmSwuN5lX7R4J_NFqmvi8vfAbm3j65VDd5GSmT7N6w-qC1iHn-pwuqy3usA4Ex2dyQ5lJEPniEuqj5Ww6yV1Uuaw/s1600/sunburst.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="260" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg18PP8gw394vBbwUbPZN0MxA0uZ3NNWiFtsrpN668rBaQzN5IlOKsqmSwuN5lX7R4J_NFqmvi8vfAbm3j65VDd5GSmT7N6w-qC1iHn-pwuqy3usA4Ex2dyQ5lJEPniEuqj5Ww6yV1Uuaw/s400/sunburst.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Circles are also pretty!</i></td></tr>
</tbody></table>
<h4 style="font-family: Times; font-size: medium;">
<span style="line-height: 115%;">
New visualization: Geo Maps</span></h4>
<span style="line-height: 115%;">The geo map uses the same auto-geocoding as Analyzer, with out of box ability to plot latitude and longitude pairs, all countries, all country subdivisions (state/province), major cities in select countries, as well as United States counties and postal codes.</span><br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAi_cdrafIj_9c1sbfUXCxRMlSJUFc74opun7-a4tLtcU5TZaT_sRoJs7MsxFz3k7VHBXPse1Xfv__W6nNRmZ6L-dbXdSrQrkimIjNlqyWFQiLPV5ZXRkSu9gr65_FZ_IRhMcT-579TH8/s1600/GeoMap.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="243" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiAi_cdrafIj_9c1sbfUXCxRMlSJUFc74opun7-a4tLtcU5TZaT_sRoJs7MsxFz3k7VHBXPse1Xfv__W6nNRmZ6L-dbXdSrQrkimIjNlqyWFQiLPV5ZXRkSu9gr65_FZ_IRhMcT-579TH8/s400/GeoMap.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Geo Map visualization</i></td></tr>
</tbody></table>
<h4 style="font-family: Times; font-size: medium;">
<span style="line-height: 115%;">
Drill down capabilities</span></h4>
<div style="font-family: Times; font-size: medium;">
<!--[if gte mso 9]><xml>
<o:OfficeDocumentSettings>
<o:RelyOnVML/>
<o:AllowPNG/>
</o:OfficeDocumentSettings>
</xml><![endif]--><span style="line-height: 115%;">
<!--[if gte mso 9]><xml>
<w:WordDocument>
<w:View>Normal</w:View>
<w:Zoom>0</w:Zoom>
<w:TrackMoves/>
<w:TrackFormatting/>
<w:PunctuationKerning/>
<w:ValidateAgainstSchemas/>
<w:SaveIfXMLInvalid>false</w:SaveIfXMLInvalid>
<w:IgnoreMixedContent>false</w:IgnoreMixedContent>
<w:AlwaysShowPlaceholderText>false</w:AlwaysShowPlaceholderText>
<w:DoNotPromoteQF/>
<w:LidThemeOther>EN-US</w:LidThemeOther>
<w:LidThemeAsian>JA</w:LidThemeAsian>
<w:LidThemeComplexScript>X-NONE</w:LidThemeComplexScript>
<w:Compatibility>
<w:BreakWrappedTables/>
<w:SnapToGridInCell/>
<w:WrapTextWithPunct/>
<w:UseAsianBreakRules/>
<w:DontGrowAutofit/>
<w:SplitPgBreakAndParaMark/>
<w:EnableOpenTypeKerning/>
<w:DontFlipMirrorIndents/>
<w:OverrideTableStyleHps/>
<w:UseFELayout/>
</w:Compatibility>
<m:mathPr>
<m:mathFont m:val="Cambria Math"/>
<m:brkBin m:val="before"/>
<m:brkBinSub m:val="--"/>
<m:smallFrac m:val="off"/>
<m:dispDef/>
<m:lMargin m:val="0"/>
<m:rMargin m:val="0"/>
<m:defJc m:val="centerGroup"/>
<m:wrapIndent m:val="1440"/>
<m:intLim m:val="subSup"/>
<m:naryLim m:val="undOvr"/>
</m:mathPr></w:WordDocument>
</xml><![endif]--><!--[if gte mso 9]><xml>
<w:LatentStyles DefLockedState="false" DefUnhideWhenUsed="false"
DefSemiHidden="false" DefQFormat="false" DefPriority="99"
LatentStyleCount="382">
<w:LsdException Locked="false" Priority="0" QFormat="true" Name="Normal"/>
<w:LsdException Locked="false" Priority="9" QFormat="true" Name="heading 1"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 2"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 3"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 4"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 5"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 6"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 7"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 8"/>
<w:LsdException Locked="false" Priority="9" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="heading 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index 9"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 1"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 2"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 3"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 4"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 5"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 6"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 7"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 8"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" Name="toc 9"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="header"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footer"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="index heading"/>
<w:LsdException Locked="false" Priority="35" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="caption"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of figures"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="envelope return"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="footnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="line number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="page number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote reference"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="endnote text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="table of authorities"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="macro"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="toa heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Bullet 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Number 5"/>
<w:LsdException Locked="false" Priority="10" QFormat="true" Name="Title"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Closing"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Signature"/>
<w:LsdException Locked="false" Priority="1" SemiHidden="true"
UnhideWhenUsed="true" Name="Default Paragraph Font"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="List Continue 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Message Header"/>
<w:LsdException Locked="false" Priority="11" QFormat="true" Name="Subtitle"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Salutation"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Date"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text First Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Heading"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Body Text Indent 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Block Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Hyperlink"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="FollowedHyperlink"/>
<w:LsdException Locked="false" Priority="22" QFormat="true" Name="Strong"/>
<w:LsdException Locked="false" Priority="20" QFormat="true" Name="Emphasis"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Document Map"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Plain Text"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="E-mail Signature"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Top of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Bottom of Form"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal (Web)"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Acronym"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Address"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Cite"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Code"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Definition"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Keyboard"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Preformatted"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Sample"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Typewriter"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="HTML Variable"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Normal Table"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="annotation subject"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="No List"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Outline List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Simple 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Classic 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Colorful 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Columns 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Grid 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table List 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table 3D effects 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Contemporary"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Elegant"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Professional"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Subtle 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Web 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Balloon Text"/>
<w:LsdException Locked="false" Priority="39" Name="Table Grid"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Table Theme"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 1"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 2"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 3"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 4"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 5"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 7"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 8"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Note Level 9"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Placeholder Text"/>
<w:LsdException Locked="false" Priority="1" QFormat="true" Name="No Spacing"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading"/>
<w:LsdException Locked="false" Priority="61" Name="Light List"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 1"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 1"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 1"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 1"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 1"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 1"/>
<w:LsdException Locked="false" SemiHidden="true" Name="Revision"/>
<w:LsdException Locked="false" Priority="34" QFormat="true"
Name="List Paragraph"/>
<w:LsdException Locked="false" Priority="29" QFormat="true" Name="Quote"/>
<w:LsdException Locked="false" Priority="30" QFormat="true"
Name="Intense Quote"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 1"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 1"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 1"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 1"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 1"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 1"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 1"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 1"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 2"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 2"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 2"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 2"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 2"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 2"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 2"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 2"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 2"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 2"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 2"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 2"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 2"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 2"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 3"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 3"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 3"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 3"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 3"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 3"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 3"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 3"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 3"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 3"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 3"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 3"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 3"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 3"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 4"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 4"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 4"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 4"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 4"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 4"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 4"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 4"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 4"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 4"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 4"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 4"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 4"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 4"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 5"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 5"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 5"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 5"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 5"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 5"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 5"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 5"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 5"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 5"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 5"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 5"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 5"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 5"/>
<w:LsdException Locked="false" Priority="60" Name="Light Shading Accent 6"/>
<w:LsdException Locked="false" Priority="61" Name="Light List Accent 6"/>
<w:LsdException Locked="false" Priority="62" Name="Light Grid Accent 6"/>
<w:LsdException Locked="false" Priority="63" Name="Medium Shading 1 Accent 6"/>
<w:LsdException Locked="false" Priority="64" Name="Medium Shading 2 Accent 6"/>
<w:LsdException Locked="false" Priority="65" Name="Medium List 1 Accent 6"/>
<w:LsdException Locked="false" Priority="66" Name="Medium List 2 Accent 6"/>
<w:LsdException Locked="false" Priority="67" Name="Medium Grid 1 Accent 6"/>
<w:LsdException Locked="false" Priority="68" Name="Medium Grid 2 Accent 6"/>
<w:LsdException Locked="false" Priority="69" Name="Medium Grid 3 Accent 6"/>
<w:LsdException Locked="false" Priority="70" Name="Dark List Accent 6"/>
<w:LsdException Locked="false" Priority="71" Name="Colorful Shading Accent 6"/>
<w:LsdException Locked="false" Priority="72" Name="Colorful List Accent 6"/>
<w:LsdException Locked="false" Priority="73" Name="Colorful Grid Accent 6"/>
<w:LsdException Locked="false" Priority="19" QFormat="true"
Name="Subtle Emphasis"/>
<w:LsdException Locked="false" Priority="21" QFormat="true"
Name="Intense Emphasis"/>
<w:LsdException Locked="false" Priority="31" QFormat="true"
Name="Subtle Reference"/>
<w:LsdException Locked="false" Priority="32" QFormat="true"
Name="Intense Reference"/>
<w:LsdException Locked="false" Priority="33" QFormat="true" Name="Book Title"/>
<w:LsdException Locked="false" Priority="37" SemiHidden="true"
UnhideWhenUsed="true" Name="Bibliography"/>
<w:LsdException Locked="false" Priority="39" SemiHidden="true"
UnhideWhenUsed="true" QFormat="true" Name="TOC Heading"/>
<w:LsdException Locked="false" Priority="41" Name="Plain Table 1"/>
<w:LsdException Locked="false" Priority="42" Name="Plain Table 2"/>
<w:LsdException Locked="false" Priority="43" Name="Plain Table 3"/>
<w:LsdException Locked="false" Priority="44" Name="Plain Table 4"/>
<w:LsdException Locked="false" Priority="45" Name="Plain Table 5"/>
<w:LsdException Locked="false" Priority="40" Name="Grid Table Light"/>
<w:LsdException Locked="false" Priority="46" Name="Grid Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="Grid Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="Grid Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="Grid Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="Grid Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="Grid Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="Grid Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="Grid Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="Grid Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="Grid Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="46" Name="List Table 1 Light"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark"/>
<w:LsdException Locked="false" Priority="51" Name="List Table 6 Colorful"/>
<w:LsdException Locked="false" Priority="52" Name="List Table 7 Colorful"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 1"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 1"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 1"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 1"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 1"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 1"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 2"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 2"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 2"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 2"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 2"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 2"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 3"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 3"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 3"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 3"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 3"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 3"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 4"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 4"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 4"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 4"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 4"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 4"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 5"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 5"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 5"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 5"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 5"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 5"/>
<w:LsdException Locked="false" Priority="46"
Name="List Table 1 Light Accent 6"/>
<w:LsdException Locked="false" Priority="47" Name="List Table 2 Accent 6"/>
<w:LsdException Locked="false" Priority="48" Name="List Table 3 Accent 6"/>
<w:LsdException Locked="false" Priority="49" Name="List Table 4 Accent 6"/>
<w:LsdException Locked="false" Priority="50" Name="List Table 5 Dark Accent 6"/>
<w:LsdException Locked="false" Priority="51"
Name="List Table 6 Colorful Accent 6"/>
<w:LsdException Locked="false" Priority="52"
Name="List Table 7 Colorful Accent 6"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Mention"/>
<w:LsdException Locked="false" SemiHidden="true" UnhideWhenUsed="true"
Name="Smart Hyperlink"/>
</w:LatentStyles>
</xml><![endif]-->
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:-536870145 1107305727 0 0 415 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:-536870145 1073786111 1 0 415 0;}
@font-face
{font-family:"MS Pゴシック";
mso-font-charset:128;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:-536870145 1791491579 134217746 0 131231 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin-top:0in;
margin-right:0in;
margin-bottom:10.0pt;
margin-left:0in;
line-height:115%;
mso-pagination:widow-orphan;
font-size:12.0pt;
mso-bidi-font-size:10.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"MS Pゴシック";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
color:black;
mso-themecolor:text1;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-size:10.0pt;
mso-ansi-font-size:10.0pt;
mso-bidi-font-size:10.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:"MS Pゴシック";
mso-fareast-theme-font:minor-fareast;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
.MsoPapDefault
{mso-style-type:export-only;
margin-bottom:10.0pt;
line-height:115%;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;
mso-header-margin:.5in;
mso-footer-margin:.5in;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
</style>
</span></div>
</div>
<span style="line-height: 115%;">
</span></div>
-->
<!--[if gte mso 10]>
<style>
/* Style Definitions */
table.MsoNormalTable
{mso-style-name:"Table Normal";
mso-tstyle-rowband-size:0;
mso-tstyle-colband-size:0;
mso-style-noshow:yes;
mso-style-priority:99;
mso-style-parent:"";
mso-padding-alt:0in 5.4pt 0in 5.4pt;
mso-para-margin-top:0in;
mso-para-margin-right:0in;
mso-para-margin-bottom:10.0pt;
mso-para-margin-left:0in;
line-height:115%;
mso-pagination:widow-orphan;
font-size:10.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;}
</style>
<![endif]-->
<!--StartFragment--><span style="font-family: "calibri"; font-size: 12pt; line-height: 115%;">When using dimensions
in Data Explorer charts or pivot tables, users can now expand hierarchies in
order to see the next level of data.
This is done by double clicking a level in the visualization (for
instance, double click a ‘country’ bar in a bar chart to drill down to ‘city’
data).</span><!--EndFragment-->
<br />
<div style="font-family: Times; font-size: medium;">
<span style="font-family: "calibri"; font-size: 12pt; line-height: 115%;"><br /></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdqc8D0Bbeke4jzZ7VVgDEDwsQlwr7APteFe-qoWoIQuPEsiT5tlYpjlExdzF9YknqpjImauvnoeR6rX3fQVjfM-0PRXekFwzZpVo-mEtWUYe_A88g73i_rxkuBIG0FS6UyvFDGupBrqs/s1600/drilldown1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="278" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhdqc8D0Bbeke4jzZ7VVgDEDwsQlwr7APteFe-qoWoIQuPEsiT5tlYpjlExdzF9YknqpjImauvnoeR6rX3fQVjfM-0PRXekFwzZpVo-mEtWUYe_A88g73i_rxkuBIG0FS6UyvFDGupBrqs/s400/drilldown1.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Drill down in the visualizations...</i></td></tr>
</tbody></table>
<div style="font-family: Times; font-size: medium;">
<br /></div>
This can be done though the visualizations or though the labels / axis. Once again, look at this as the beginning of a coherent way to handle data exploration!<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7d0Y38Q9CKznzHBIFe8So4X7sEURzvea9HmGiSWuc0HlblNEhsvwbmYgdapIjJE1WsVPXGLqpfrbEaELh9lX0mg-C6WQdi49VrNnz2NULMv6NCBYWrAvJxizWda5pq9SwTm6k2W3PNPk/s1600/drilldown2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="280" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh7d0Y38Q9CKznzHBIFe8So4X7sEURzvea9HmGiSWuc0HlblNEhsvwbmYgdapIjJE1WsVPXGLqpfrbEaELh9lX0mg-C6WQdi49VrNnz2NULMv6NCBYWrAvJxizWda5pq9SwTm6k2W3PNPk/s400/drilldown2.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>... or from where it makes more sense</i></td></tr>
</tbody></table>
And this is only the first of a new set of <i>actions</i> we'll introduce here...<br />
<h4>
<span style="font-family: "calibri";">Analysis persistency</span></h4>
In 7.0 these capabilities were a one-time inspection only. Now we've taken a step further - they get persisted with the transformations. You can now use to validate the data, get insights right on the spot, and make sure everything is lined up to show to the business users.<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiagWX_7R3w-yhu4yrYwaQLLeojBv0TFPOmQTYNsLlHzjBHg-tR6TMKphUd3alfjWuwZCYpCAWotQe9NqWylaXHI5pkZx_0AVUhgd2lTR06Cz5m7KQfRjO1xDQuGB1hZ20KgikgNtb7k7c/s1600/tabPersistency.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="205" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiagWX_7R3w-yhu4yrYwaQLLeojBv0TFPOmQTYNsLlHzjBHg-tR6TMKphUd3alfjWuwZCYpCAWotQe9NqWylaXHI5pkZx_0AVUhgd2lTR06Cz5m7KQfRjO1xDQuGB1hZ20KgikgNtb7k7c/s400/tabPersistency.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Analysis persistency indicator</i></td></tr>
</tbody></table>
<h3>
<span style="font-family: "calibri";">Viz Api 3.0</span></h3>
Every old timer knows how much disparity we've had throughout the stack in terms of offering a consistent visualization. This is not an easy challenge to solve - the reason they are different is because different parts of our stack were created in completely different times and places, so a lot of different technologies were used. An immediate follow-up consequence is that we can't just add a new viz and expect it to be available in several places of the stack<br />
<br />
We're been working on a visualization layer, codenamed VizAPI (for a while, actually, but now we reached a point where we can make it available on beta form), that brings this so needed consistency and consolidation.<br />
<div>
<span style="font-family: "calibri";"><br /></span></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibY4ASymxYdM8JXiveurvH0kLOPOYEdKya7lobhnEn97NyJ_UUlyyPbcUIvZRg3JvxxMUUAyO5sPznBlNL_S7sEHjGAmOl5uY3QPh21QqDua6eYfIwei-NDDLreqN4B5dRWNVB1Th9YHw/s1600/vizApiPentahoServer.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="190" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEibY4ASymxYdM8JXiveurvH0kLOPOYEdKya7lobhnEn97NyJ_UUlyyPbcUIvZRg3JvxxMUUAyO5sPznBlNL_S7sEHjGAmOl5uY3QPh21QqDua6eYfIwei-NDDLreqN4B5dRWNVB1Th9YHw/s400/vizApiPentahoServer.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Viz API compatible containers</i></td></tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
In order to make this effort worthwhile, we needed the following solve order:<br />
<br />
<ol>
<li>Define the VizAPI structure</li>
<li>Implement the VizAPI in several parts of the product</li>
<li>Document and allow users to extend it</li>
</ol>
<br />
<br />
And... we did it. We re-implemented all the visualizations in this new VizAPI structure, adapted 3 containers - Analyzer, Ctools and DET (Data Exploration) in PDI, and as a consequence, the look and feel of the visualizations are the same<br />
<div>
<br /></div>
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6VlsxExeKYPUc7uEZCrLWxOGTklq_3yoTIah_h-iVHRzxIGLLTZa_go2hUDt85A8a4e-T0HuyS_H1a9WxO0wAl29JNII3g-5KKtSMJPkyNho3e1vHlWAWIGLSJP7eNbkf8jnC-M22Qd0/s1600/VizApiAnalyzer.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="187" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh6VlsxExeKYPUc7uEZCrLWxOGTklq_3yoTIah_h-iVHRzxIGLLTZa_go2hUDt85A8a4e-T0HuyS_H1a9WxO0wAl29JNII3g-5KKtSMJPkyNho3e1vHlWAWIGLSJP7eNbkf8jnC-M22Qd0/s400/VizApiAnalyzer.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Analyzer visualizations are now much better looking _and_ usable</i></td></tr>
</tbody></table>
<div>
<span style="font-family: "calibri";"><br /></span></div>
One important note though - migration users will still default to the "old" VizAPI (yeah, we called it the same as well, isn't that smart :/ ) not to risk interfering with existing installations. In order for you to test an existing project with the new visualizations you need to change the VizAPI version number in analyzer.properties. New installs will default to the new ones. <br />
<br />
In order to allow people to include their own visualization and promote more contributions to Pentaho (I'd love to start seeing more contributions to the marketplace with new and shiny Viz's), we need to really make it easy for people to know how to create them.<br />
<br />
And I think we did that! Even though this will require it's own blog post, just <a href="http://pentaho.github.io/pentaho-platform-plugin-common-ui/platform/visual/">take a look at the documentation the team prepared</a> for this<br />
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu18mq7PTjkwN-zIIs8eGmvnbfrGy1RxdolRbIoNfBwQ2C1_u_3FHrvFlwD424deVC4C1WTABxFDtsOzWAjm_mQ1UjDEPRkFDeZVAE9iyLIeoPGyJTR37RMscWpfWU8qqRliapdykC1v0/s1600/VizAPIDoc.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="358" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiu18mq7PTjkwN-zIIs8eGmvnbfrGy1RxdolRbIoNfBwQ2C1_u_3FHrvFlwD424deVC4C1WTABxFDtsOzWAjm_mQ1UjDEPRkFDeZVAE9iyLIeoPGyJTR37RMscWpfWU8qqRliapdykC1v0/s400/VizAPIDoc.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Instructions for how to add new visualizations</i></td></tr>
</tbody></table>
<div class="MsoNormal" style="font-family: Times; font-size: medium; line-height: 18.4px;">
<br /></div>
You'll see this documentation has beta written on it. The reason is simple - we decided to put it out there, collect feedback from the community and implement any changes / fine tunes / etc before 8.0 timeframe, where we'll lock this down, guaranteeing long term support for new visualizations<br />
<div class="MsoNormal" style="font-family: Times; font-size: medium; line-height: 18.4px;">
<br /></div>
<div>
<h4>
MS HD Insights</h4>
HD Insights (HDI) is a hosted Hadoop cluster that is part of Microsoft’s Azure cloud offering. HDI is based on Hortonworks Data Platform (HDP). One of the major differences between the standard HDP release and HDI’s offering is the storage layer. HDI connects to local cluster storage via HDFS or to Azure Blob Storage (ABS) via a WASB protocol.</div>
<div>
<br /></div>
<div>
We now have a shim that allows us to leverage this cloud offering, something we've been seeing getting more and more interest on the marketplace.</div>
</div>
<div>
<br /></div>
<br />
<div>
<!--EndFragment--></div>
<h3>
Hortonworks security support</h3>
<div>
This is a continuation of the previous release, available on the Enterprise Edition (EE)</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://2.bp.blogspot.com/7XV92VHupQxRzt17YFWiWZKlx1CuXn0-mXXSBlZDCU5JJbGpNA71StONjYVOksRTo145DGxD-dpWSKDfBnPLz31-ITChFDoaWotfCsKLxVDJxte3e5bUt_A3NkBoFNiPZNSTExAWV9NCWy_dAA" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="138" src="https://2.bp.blogspot.com/7XV92VHupQxRzt17YFWiWZKlx1CuXn0-mXXSBlZDCU5JJbGpNA71StONjYVOksRTo145DGxD-dpWSKDfBnPLz31-ITChFDoaWotfCsKLxVDJxte3e5bUt_A3NkBoFNiPZNSTExAWV9NCWy_dAA" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Added support for Hadoop user impersonation</i></td></tr>
</tbody></table>
<div>
Earlier releases of PDI introduced enterprise security for Cloudera, specifically, Kerberos Impersonation for authentication and integration with Apache Sentry for authorization. </div>
<div>
<br /></div>
<div>
This release of PDI extends these enterprise level security features to Hortonworks’s Hadoop distribution as well. Kerberos Impersonation is now support Hortonworks’s HDP. For authorization, PDI integrates with <a href="http://ranger.apache.org/">Apache Ranger</a>, an alternative OSS component included in the HDP security platform.</div>
<div>
<br /></div>
<div>
<h3>
Data Processing-Enhanced Spark Submit and SparkSQL JDBC</h3>
</div>
<div>
<div>
Earlier PDI and BA/Reporting releases broaden access to Spark for querying and preparing data through a dedicated transformation step Spark Submit and Spark SQL JDBC. </div>
<div>
<br /></div>
<div>
This release will be extending these existing features to support additional vendors so that these features can be used more widely. Apart from additional vendors, these features have been now certified with a more up to date version of Spark 2.0. </div>
<div>
<br /></div>
<div>
Additional big data infrastructure vendors supported for these functionalities apart from Cloudera and Hortonworks:</div>
<div>
<ol>
<li>Amazon EMR</li>
<li>MapR</li>
<li>Azure HD Insights</li>
</ol>
<div>
<h3>
</h3>
<h3>
VCS Improvements</h3>
<div>
<h4>
Repository agnostic transformations and jobs</h4>
<div>
Currently some specific step interfaces (the sub-transformation one being the more impactful) where the ETL dev has to choose, upfront, if he's using a file on the file system or the repository. This prevents us from being able to abstract the environment where we're working, so checking out things from git/svn and just import them is a no-go.</div>
</div>
<div>
<br /></div>
<div>
Here's an example of a step that used this:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieQlvlq0W2A1VagHwYzaVpzI1uttc8R11y6ctVT7wPnc2kN4osi2qR9ra2W6uk3PmNaaWvJjMPKbLHBvMeAnuAfZh5wluUgX97Y5f7tvqnsdWsgKVZ7uWtlhSrFfRYZ4zMgiUVAm_8RHg/s1600/vcs-old.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="233" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEieQlvlq0W2A1VagHwYzaVpzI1uttc8R11y6ctVT7wPnc2kN4osi2qR9ra2W6uk3PmNaaWvJjMPKbLHBvMeAnuAfZh5wluUgX97Y5f7tvqnsdWsgKVZ7uWtlhSrFfRYZ4zMgiUVAm_8RHg/s400/vcs-old.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The classic way to reference dependent objects</i></td></tr>
</tbody></table>
<div>
ThisIn general, we need to abstract the linkage to other artifacts (sub-jobs and sub-transformations) independent on the used repository or file system.</div>
<div>
<br /></div>
<div>
The linkage needs to work in all environments whether it is a repository (Pentaho, Database, File) or File Based system (kjb and ktr).</div>
<div>
<br /></div>
<div>
The linkage needs to work independently of the execution system: On the Pentaho Server, on a Carte Server (with a repository or file based system), in Map Reduce and future execution systems as part of the Adaptive Execution System (AES) </div>
<div>
<br /></div>
<div>
So we turned this into something much simpler:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBwZ9HH1wgY1EIZ6p37ymhCD8SCM8UQNDUekwUshzkl7BzzfbY4CdVVxB5RG2WbewjtuBmVlQR7QR7z0PtwTniGUEqJrqJCygueRGDPnjq2vGRvtBANi-8NEi22SkLSLZkvrLM91P-zyU/s1600/vfs.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="376" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjBwZ9HH1wgY1EIZ6p37ymhCD8SCM8UQNDUekwUshzkl7BzzfbY4CdVVxB5RG2WbewjtuBmVlQR7QR7z0PtwTniGUEqJrqJCygueRGDPnjq2vGRvtBANi-8NEi22SkLSLZkvrLM91P-zyU/s400/vfs.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>The current approach to define dependencies</i></td></tr>
</tbody></table>
<div>
We just define where the transformation lives. This may seem a <i>"what, just this??"</i> moment, but now we can just work locally, remotely, check into a repository, even automate the promotion and control the lifecycle in between different installation environments. I'm absolutely sure that existing users will value this a lot (as we can deprecate the stupid file-based repository)</div>
<div>
<br /></div>
<h4>
KTR / KJB XML format</h4>
<div>
We did something very simple (in concept), but very useful. While we <i>absolutely don't recommend</i> playing around with the job and transformation files (they are plain old XML files), we guaranteed that they are properly indented. Why? Cause when we use a version control system (git / svn, don't care which as long as you USE one!), you can easily identify what changes happened from version to version</div>
<div>
<br /></div>
<h4>
Repository performance improvements</h4>
<div>
We want you to use the Pentaho Repository. And till now, performance while browsing that repository from Spoon was crap (there's no other way to say it!). We addressed that - it's now about 100x faster to browse and open files from the repository</div>
<div>
<br /></div>
<h3>
Operations Mart Updates</h3>
<div>
Also known as the ops marts, available in EE. Used to work. Then it stoped working. Now it's working again. Yay :/ </div>
<div>
<br /></div>
<div>
I'll skip this one. I hate it. We're working on a different way to handle monitoring on our product, and at scale</div>
<div>
<br /></div>
<h3>
Other Data Integration Improvements</h3>
<div>
Apart from all the above new big features, there are some smaller data integration enhancements added to product to build data pipeline with Pentaho easier.</div>
<div>
<br /></div>
<h4>
Metadata Injection Enhancement</h4>
<div>
Metadata Injection enables creating generalized ETL transformations whose behavior can be changed at run-time and thus significantly improves data integration developer agility and productivity. </div>
<div>
In this release, a new option for constant has been added for Metadata Injection which will help making steps more dynamic with Metadata Injection feature. </div>
<div>
This functionality extended to Analytic Query and Dimension Lookup/Update steps which will help making these steps dynamic and thus make them highly dynamic. Dynamism of these steps will improve the Data Warehouse & Customer 360 blueprints and similar analytic data pipeline. </div>
<div>
<br /></div>
<h4>
Lineage Collection Enhancement</h4>
<div>
Customers can now configure the location for the lineage output and add the ability to write to VFS location. This will help customers to maintain lineage in clustered / transient node environments, such as Pentaho MapReduce. Lineage information helps with data compliance and security needs of the customers. </div>
<div>
<br /></div>
<h4>
XML Input Step Enhancement</h4>
<div>
XML Input Stream (StAX) step has been updated to receive XML from a previous step. This will make it easier to develop XML processing in data pipeline when you are working with XML data. </div>
</div>
</div>
</div>
<div>
<br /></div>
<h3>
New Mobile approach (and the deprecation of Pentaho Mobile)</h3>
<div>
<div>
We used to have a mobile specific plugin, introduced in a previous Pentaho release, that enabled touch gestures to work with analyzer.</div>
<div>
<br /></div>
<div>
But while it sounds good, in fact it didn't work as we'd expected. The fact that we had to develop and maintain a completely separate access to information caused that mobile plugin to become very outdated. </div>
<div>
<br /></div>
<div>
To complement that, the maturity of the browsers on mobile devices and the increased strength of tables makes it possible for Pentaho reports and analytic views to be accessed directly without any specialized mobile interface. Thus, we are deprecating the Pentaho mobile plug-in and investing on the responsive capabilities of the interface</div>
</div>
<div>
<br /></div>
<div>
It sounds bad? Actually it's not - just use your tablet to access your EE pentaho, looks great :)</div>
<div>
<br /></div>
<h3>
Pentaho User Console Updates</h3>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmMJeCt0SX-atTjLSITNj8OGoW0Il21G4Nltw1vkoHitwgippDMoQCyE9xx4rL7VzX7GYtQohzq69c9hpg7AZV2tj69Ex4ZaNmnIdl5W8jY5eGOE3M_FuCirD3-Mwpiwo2ns2mFP5UIJk/s1600/sapphire.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="252" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgmMJeCt0SX-atTjLSITNj8OGoW0Il21G4Nltw1vkoHitwgippDMoQCyE9xx4rL7VzX7GYtQohzq69c9hpg7AZV2tj69Ex4ZaNmnIdl5W8jY5eGOE3M_FuCirD3-Mwpiwo2ns2mFP5UIJk/s400/sapphire.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Sapphire theme in PUC</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
<div>
Starting in Pentaho 7.1, Onyx will be deprecated and removed from the list of available themes in PUC. In addition, a new theme “Sapphire” has been introduced in 7.0. As of Pentaho 7.1, Sapphire will be PUC’s default selected theme. Crystal will be the available alternative.</div>
<div>
<br /></div>
<div>
Moreover, a newly refreshed log-in screen has been implemented in Pentaho 7.1, this screen has been based on the new Sapphire theme that was introduced in Pentaho 7.0. This is something that was already in 7.0 CE and now it's the default for EE as well</div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
-----------------------</div>
<div>
<br />
As usual, you can get EE from <a href="http://www.pentaho.com/download">here</a> and CE from <a href="http://community.pentaho.com/">here</a><br />
<br />
<br /></div>
<div>
This is a spectacular release! I should be celebrating! But instead, it's 8pm, I'm stuck in the office writing this blog post, and already very very stressed because I have all my 8.0 work stuff already piling up on my inbox... :(</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
I'm out, have fun!</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
-pedro</div>
<div>
<br /></div>
<div>
<br /></div>
<br />Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com7tag:blogger.com,1999:blog-6644329693530300467.post-14646814643623398512017-04-19T14:57:00.000+01:002017-04-19T14:57:00.911+01:00PentahoDay 2017 - Brazil, Curitiba, May 11 and 12<div class="separator" style="clear: both; text-align: center;">
</div>
<h3>
PentahoDay 2017 - Brazil, Curitiba, May 11 and 12</h3>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRQXvpu0wdMnmhK8IV3Tvd7ZOwsb7LVYD-u3dGfuIKDg5y7XS2JMSE23phWqDYA2dmy-5U_o4R4DffOkLaOvjeb-2gpRaIGTqeeAJddgzf0Bzog79wulhjnHvQrQtepErS8MY3u02D6fo/s1600/header.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="203" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhRQXvpu0wdMnmhK8IV3Tvd7ZOwsb7LVYD-u3dGfuIKDg5y7XS2JMSE23phWqDYA2dmy-5U_o4R4DffOkLaOvjeb-2gpRaIGTqeeAJddgzf0Bzog79wulhjnHvQrQtepErS8MY3u02D6fo/s640/header.png" width="640" /></a></div>
<br />
After a pause to rest in 2016, the <a href="http://www.pentahobrasil.com.br/eventos/pentahoday2017/">biggest Pentaho event organized by the community is back</a>. 2 days, May 11 and 12, dozens of presentations, use cases, even hands-on mini-labs will happen in Curitiba, Brazil.<br />
<br />
<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE0xkl04U8Qx_9pQ1pjt168YhDgpTMH4T5G9ZB9sss52-kRETspCw0m3w36kNz2AOfGm5b9ie6xNQ66ZWpmKcJUvUjpJeYY57ujN5RG-jxdjysr8wyXNoLqQlEX-7jjf9_JqDQQ0gGjSE/s1600/speakers.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="285" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgE0xkl04U8Qx_9pQ1pjt168YhDgpTMH4T5G9ZB9sss52-kRETspCw0m3w36kNz2AOfGm5b9ie6xNQ66ZWpmKcJUvUjpJeYY57ujN5RG-jxdjysr8wyXNoLqQlEX-7jjf9_JqDQQ0gGjSE/s400/speakers.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Pentaho Day speakers</i></td></tr>
</tbody></table>
<br />
400 attendees or more are expected on this huge event. It's really amazing, so if you're even near South America, be there!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcEZwhMd7zoLST5YDm0k67ZeBLrvjPq5_S5viq2mAi_Zr7S1e8kQcmutI9ApgFJcmSVvlqngb-F01YkVF9nRzkJh3NGbF0m3zZ7z8lSLv-5_TzjLoMCiYuxhpc049QzE6pdxKiPKVUiks/s1600/location.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="235" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgcEZwhMd7zoLST5YDm0k67ZeBLrvjPq5_S5viq2mAi_Zr7S1e8kQcmutI9ApgFJcmSVvlqngb-F01YkVF9nRzkJh3NGbF0m3zZ7z8lSLv-5_TzjLoMCiYuxhpc049QzE6pdxKiPKVUiks/s400/location.png" width="400" /></a></div>
<br />
<br />
<br />
<b><a href="https://www.eventbrite.com/e/pentaho-day-2017-universidade-positivo-tickets-32393041459">Register here</a></b><br />
<br />Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com1tag:blogger.com,1999:blog-6644329693530300467.post-39932473138016787842017-04-03T11:58:00.000+01:002017-04-03T12:14:54.105+01:00Building Pentaho Platform from source and debugging it<h3>
After all, if it's open source, means we can compile it, right?</h3>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blog.storeya.com/wp-content/uploads/2015/05/I-love-this-hammer.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="320" src="https://blog.storeya.com/wp-content/uploads/2015/05/I-love-this-hammer.png" width="320" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>I'm sure you've guessed by now this is not an original image from me even though I've been told I'm very good at drawing stuff - and I always believe my daughter!</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
Sure - but sometimes it's not as easy as it seems. However, <a href="http://www.pentaho.com/">we're</a> doing a huge consolidation work to streamline all our build processes. Historically, each project, specially the older ones (kettle, mondrian, prd, ctools) used each own build method, depending on the author's personal stance on them (and boy, there are some heavy opinions in here...)</div>
<div>
<br /></div>
<div>
Personally, I come from the <i>CCLJTMHIWAPMIS</i> school of thought (for the ones not familiar with it, the acronym means <i>Couldn't Care Less Just Tell Me How It Works And Please Make It Simple</i>, very popular specially within lazy Portuguese people).</div>
<div>
<br /></div>
<div>
And we're now doing this, slowly and surely, to all projects, as you can see from browsing through <a href="https://github.com/pentaho/">Pentaho's Github</a>.</div>
<div>
<br /></div>
<div>
So let's take a look at an example - building <a href="https://github.com/pentaho/pentaho-platform">Pentaho Platform</a> from source. Please note that we'll try to make sure the project's README.md contains the correct instructions. Also, this won't work for all versions, as we don't backport this changes; In the case of Pentaho Platform, this works for master and will appear in 7.1. Other will have it's own timeline.</div>
<div>
<br /></div>
<h3>
Compiling Pentaho Platform</h3>
<h4>
1. Clone it from source</h4>
<div>
Ok, so step one, clone it from source:</div>
<div>
<br /></div>
<blockquote class="tr_bq">
$ git clone https://github.com/pentaho/pentaho-platform.git</blockquote>
<br />
(or use git:// if you already have a user)<br />
<h4>
2. Set up your m2 config right</h4>
<div>
Before compiling it, you need to set some stuff in your maven settings file. In your home directory, under the <i>.m2</i> folder, place <a href="https://raw.githubusercontent.com/pentaho/maven-parent-poms/master/maven-support-files/settings.xml">this settings</a> file. If you already one m2 settings files, that means you're probably familiar with maven in the first place and will know how to merge the two. Don't ask me, I have no clue.</div>
<div>
<br /></div>
<div>
If you're wondering why we need a specific settings file... I wonder too, but since my laziness is bigger than my curiosity (<i>CCLJTMHIWAPMIS</i>, remember?) I think I zoned out when they were explaining it to me and now I forgot.</div>
<h4>
3. Build it</h4>
<div>
This one is easy :)</div>
<blockquote class="tr_bq">
$ mvn clean install</blockquote>
<div>
or the equivalent without the tests:</div>
<div>
<blockquote class="tr_bq">
$ mvn clean package -Dmaven.test.skip=true</blockquote>
<div>
If all goes well, you should see </div>
</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<div>
<span style="font-size: xx-small;">[INFO]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] --- maven-site-plugin:3.4:attach-descriptor (attach-site-descriptor) @ pentaho-server-ce ---</span></div>
<div>
<span style="font-size: xx-small;">[INFO]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] --- maven-assembly-plugin:3.0.0:single (assembly_package) @ pentaho-server-ce ---</span></div>
<div>
<span style="font-size: xx-small;">[INFO] Building zip: /Users/pedro/tex/pentaho/pentaho-platform-master/assemblies/pentaho-server/target/pentaho-server-ce-7.1-SNAPSHOT.zip</span></div>
<div>
<span style="font-size: xx-small;">[INFO] ------------------------------------------------------------------------</span></div>
<div>
<span style="font-size: xx-small;">[INFO] Reactor Summary:</span></div>
<div>
<span style="font-size: xx-small;">[INFO]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] Pentaho BI Platform Community Edition .............. SUCCESS [ 4.461 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-platform-api ............................... SUCCESS [ 10.149 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-platform-core .............................. SUCCESS [ 19.819 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-platform-repository ........................ SUCCESS [ 2.210 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-platform-scheduler ......................... SUCCESS [ 0.172 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-platform-build-utils ....................... SUCCESS [ 1.695 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-platform-extensions ........................ SUCCESS [01:22 min]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-user-console ............................... SUCCESS [ 19.596 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] Platform assemblies ................................ SUCCESS [ 0.059 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-user-console-package ....................... SUCCESS [ 16.399 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-samples .................................... SUCCESS [ 1.159 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-plugin-samples ............................. SUCCESS [ 11.129 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-war ........................................ SUCCESS [ 45.434 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-style ...................................... SUCCESS [ 0.742 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-data ....................................... SUCCESS [ 0.211 s]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-solutions .................................. SUCCESS [31:31 min]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-server-manual-ce ........................... SUCCESS [01:15 min]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] pentaho-server-ce .................................. SUCCESS [01:51 min]</span></div>
<div>
<span style="font-size: xx-small;">[INFO] ------------------------------------------------------------------------</span></div>
<div>
<span style="font-size: xx-small;">[INFO] BUILD SUCCESS</span></div>
<div>
<span style="font-size: xx-small;">[INFO] ------------------------------------------------------------------------</span></div>
<div>
<span style="font-size: xx-small;">[INFO] Total time: 38:36 min</span></div>
<div>
<span style="font-size: xx-small;">[INFO] Finished at: 2017-03-31T15:36:43+01:00</span></div>
<div>
<span style="font-size: xx-small;">[INFO] Final Memory: 102M/1084M</span></div>
<div>
<span style="font-size: xx-small;">[INFO] ------------------------------------------------------------------------</span></div>
</div>
<br />
There you go! In the end you should see a dist file like <i>assemblies/pentaho-server/target/pentaho-server-ce-<version>-SNAPSHOT.zip.</version></i> Unzip it, run it, done.<br />
<br />
<h3>
Debugging / inspecting the code</h3>
<div>
So the next thing you'd probably want, would be to be able to inspect and debug the code. This is actually pretty simple and common to all java projects. Goes something like this:</div>
<div>
<br /></div>
<h4>
1. Open the project in a Java IDE</h4>
<div>
Since we use maven, it's pretty straightforward to do this - simply navigate to the folder and open the project as a maven project.</div>
<div>
<br /></div>
<div>
In theory, any java IDE would do, but I had some issues with <a href="https://netbeans.org/">Netbeans</a> given it uses an outdated version of maven and ended up switching to <a href="https://www.jetbrains.com/idea/">IntelliJ IDEA</a>.</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKBaTreNjpaDCp4_h_ZXHOa-tUeIMDVzig86BFofz4ZZvzrmCqHrRQetRl-qMin2RmSJpFsA-NoVBrHopUlXl6rv4UHWVubNQnoHKO4PrTcXWhyCGiSoI98FARp9164z97GRyTNjxd5Ck/s1600/idea.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="266" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiKBaTreNjpaDCp4_h_ZXHOa-tUeIMDVzig86BFofz4ZZvzrmCqHrRQetRl-qMin2RmSJpFsA-NoVBrHopUlXl6rv4UHWVubNQnoHKO4PrTcXWhyCGiSoI98FARp9164z97GRyTNjxd5Ck/s400/idea.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>I actually took this screenshot of IntelliJ myself, so no need to give credits to anyone</i></td></tr>
</tbody></table>
<div>
<br /></div>
<h4>
2. Define a remote run configuration</h4>
<div>
Now you need to define a remote debug configuration. It works pretty much the same in all IDEs. Make sure you point to the port of the Java Debug Wire Protocol (JDWP) port you'll be using in the application you're attaching to</div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyLalRel0K69UjLR8v9T3UUBlp9FBxU5Sci7ofFVowhon2Ysd-cPNlxGC5lMY7Udk0dXpxnrdO-sMZJMLj0G3zzHUfRfJexRA0J55sMZHy4ynEk4sZEh8QeoZI2Qhf41A81U10Z7cTaGw/s1600/debugConfig.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="265" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjyLalRel0K69UjLR8v9T3UUBlp9FBxU5Sci7ofFVowhon2Ysd-cPNlxGC5lMY7Udk0dXpxnrdO-sMZJMLj0G3zzHUfRfJexRA0J55sMZHy4ynEk4sZEh8QeoZI2Qhf41A81U10Z7cTaGw/s400/debugConfig.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Setting up a debug configuration</i></td></tr>
</tbody></table>
<h4>
3. Make sure you start your application with JDWP enabled</h4>
<div>
This sounds complex, but really isn't. Just make sure your java command includes the following options:</div>
<blockquote class="tr_bq">
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8044</blockquote>
<div>
For pentaho platform is even easier, as you can simply run <i>start-pentaho-debug.sh</i>. </div>
<div>
<br /></div>
<h4>
4. Once the server / application is running, simply attach to it</h4>
<div>
And from this point on, any breakpoints should be intercepted</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1fB4tvtj7K18BMyJ1Vbo9gd0ISDbaF_wgXh7HJsuok57s6Uqqf4NIHyajDySy4bdNM_P27ZgQjYZOa0bDhUo3o3nkKKmIf8zBl4CIC_btqgVY5mPepFMJ_3taGs0YeGCq3Ad1ZmWGYMA/s1600/breakpoints.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="220" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1fB4tvtj7K18BMyJ1Vbo9gd0ISDbaF_wgXh7HJsuok57s6Uqqf4NIHyajDySy4bdNM_P27ZgQjYZOa0bDhUo3o3nkKKmIf8zBl4CIC_btqgVY5mPepFMJ_3taGs0YeGCq3Ad1ZmWGYMA/s400/breakpoints.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Inspecting and debugging the code</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
<h3>
Submitting your fixes</h3>
<div>
Now that you know how to compile and debug code, we're a contributor in the works! Let's imagine you add some new functionality or fix a bug, and you want to send it back to us (you do, right???). Here are the steps you need - they may seem extensive but it's really pretty much the normal stuff:<br />
<ol>
<li>Create a <a href="http://jira.pentaho.com/">jira</a></li>
<li>Clone the repository</li>
<li>Implement the improvement / fixes in your repository</li>
<li>Make sure to include a unit test on it (link to how to write a unit test or a sample would be good)</li>
<li>Separate formatting-only commits from actual commits. So if your commit reformats the java class, you need to have a commit with <i>[CHECKSTYLE]</i> as the commit comment. Your main changes including your test case should be in a single commit. </li>
<li>Get the <a href="https://github.com/pentaho/pentaho-coding-standards">code formatting style template for your IDE</a> and update the year in the copyright header</li>
<li>Issue a pull request against the project with <i>[JIRA-ID]</i> as the start of the commit comment</li>
<li>For visibility, add that PR to the jira you created, email me, tweet, whatever it takes. Won't promise it will be fast, but I promise we'll look :)</li>
</ol>
</div>
</div>
<br />
Hope this is useful!<br />
<br />
<br />
-pedro<br />
<br />Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com6tag:blogger.com,1999:blog-6644329693530300467.post-25797798944712038332017-01-26T17:29:00.002+00:002017-01-26T17:29:34.997+00:00[Marketplace Spotlight] BTable 3.x<br />
Marketplace spotlight time! This time for an amazing contribution by our Italian friends from <a href="http://www.biztech.it/">Biztech.it</a>.<br />
<br />
<a href="https://twitter.com/bonomma">Massimo Bonometto</a> just blogged about the new BTable release, that I shamelessly report here:<br />
<br />
<br />
Hats off, Massimo!<br />
<br />
<br />
__________________________________________________<br />
<a href="http://bonomma.blogspot.pt/2017/01/announcing-btable-3x.html"><br /></a>
<h3>
<a href="http://bonomma.blogspot.pt/2017/01/announcing-btable-3x.html">Repost from Massimo's blog post</a></h3>
<br />
<br />
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;">In January 2017 a new BTable version has been released to Pentaho Community.</span></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;">As always it is available from Pentaho Markeplace.</span></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;"><br /></span></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;"><b>Note about BTable version numbering</b>: Pentaho 7.0 uses an newer version of Spring platform. This is why we are forced to maintain 2 different versions of BTable. BTable 3.0 works with Pentaho 5.x and 6.x while BTable 3.6 is the one for Pentaho 7.x. </span></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<h3 style="font-weight: bolder; margin: 0.83em 0px; padding: 0px; position: relative;">
<span style="color: orange; font-family: arial, helvetica, sans-serif;">What's New?</span></h3>
</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;">In the following I'm going to give a brief description of the most important features introduced with this new version. </span><br /><span style="font-family: arial, helvetica, sans-serif;"><br /></span><span style="color: orange; font-family: arial, helvetica, sans-serif;">Styling And Alarms</span></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;">We introduced the concept of BTable Templates. One template is a JSON file with .bttemplate suffix, that usually lives inside Pentaho Repository, whose structure is composed of 3 sections:</span><br /><ul style="list-style-image: initial; list-style-position: initial; margin: 0.5em 0px; outline: none; padding: 0px 0px 0px 2em;">
<li style="margin: 0px; outline: none; padding: 0px;"><span style="font-family: arial, helvetica, sans-serif;">alarmRules: defines the alarm logic for each measure;</span></li>
<li style="margin: 0px; outline: none; padding: 0px;"><span style="font-family: arial, helvetica, sans-serif;">inlineCss: contains CSS statements added dynamically to one single BTable;</span></li>
<li style="margin: 0px; outline: none; padding: 0px;"><span style="font-family: arial, helvetica, sans-serif;">externalCss: similar to the previous one but uses externalCss file. </span></li>
</ul>
<div class="separator" style="clear: both; margin: 0px; outline: none; padding: 0px; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcse-voYAjaRxg2TV4p4rKQLngScJO_pAdR4hPOctUt73OvwzgOmJY8uVYetD7azwFnBziKs4aibWVY9fuxZN4zaoiSR-8BCnXplzBiLCbBrefk72YT-MkcrdlK5FzceMHDeXK72YUSkuH/s1600/BTable_With_Templates.png" imageanchor="1" style="color: #009eb8; display: inline; margin-bottom: 0px !important; margin-left: 1em; margin-right: 1em; margin-top: 0px !important; outline: none; text-decoration: none; transition: color 0.3s;"><img border="0" class="" height="223" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcse-voYAjaRxg2TV4p4rKQLngScJO_pAdR4hPOctUt73OvwzgOmJY8uVYetD7azwFnBziKs4aibWVY9fuxZN4zaoiSR-8BCnXplzBiLCbBrefk72YT-MkcrdlK5FzceMHDeXK72YUSkuH/s320/BTable_With_Templates.png" style="-webkit-border-image: url("data:image/png; border: 9px none; box-sizing: border-box; display: inline-block; height: auto; margin: 10px auto; max-width: 100%; padding: 8px; position: relative;" width="320" /></a></div>
<br /><span style="font-family: arial, helvetica, sans-serif;">Alarm styling is based on CSS and gives developers the opportunity to create very nice results. </span><br /><span style="font-family: arial, helvetica, sans-serif;"><br /></span><span style="font-family: arial, helvetica, sans-serif;">The Template is a BTable property and can be set inside CDE or changed in BTable Analyzer; that is developers can create, for example, many templates with different alarm logics and users can dynamically change templates in order to evaluate their effect.</span><br /><span style="font-family: arial, helvetica, sans-serif;"><br /></span><span style="font-family: arial, helvetica, sans-serif;">It is possible to drive the default template for all BTables and default template for each Mondrian cube. Just create a new folder named /public/BTableCustom and add:</span><br /><br /><ul style="list-style-image: initial; list-style-position: initial; margin: 0.5em 0px; outline: none; padding: 0px 0px 0px 2em;">
<li style="margin: 0px; outline: none; padding: 0px;"><span style="font-family: arial, helvetica, sans-serif;">Default_<i>Mondrian</i> <i>Catalog_Mondrian Cube</i>.bttemplate (For example Default_SteelWheels_SteelWheelsSales): it is used as default for BTables on specific Mondrian cube;</span></li>
<li style="margin: 0px; outline: none; padding: 0px;"><span style="font-family: arial, helvetica, sans-serif;">Default.bttemplate: it is used as default when a specific template for cube is not found.</span></li>
</ul>
<br /><div style="margin: 0px; outline: none; padding: 0px;">
<div style="margin: 0px; outline: none; padding: 0px;">
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="color: orange; font-family: arial, helvetica, sans-serif;">Show Table Option </span></div>
</div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">I'm sure that most of you love to spend time adding filters to CDE dashboards. Well, I really hate it!!! (In particular when a customer asks to add one filter after I finished the dashboard). </span><span style="font-family: arial, helvetica, sans-serif;">:-)</span><span style="font-family: arial, helvetica, sans-serif;"> </span><br /><span style="font-family: arial, helvetica, sans-serif;">This is why I had the idea to use BTable just for filters selection. I find it really tricky.</span><br /><span style="font-family: arial, helvetica, sans-serif;"><br /></span><span style="font-family: arial, helvetica, sans-serif;">In the BTable With Templates example I show you how you can add a BTable just for filter selection and then synchronize other 2 BTables.</span><br /><span style="font-family: arial, helvetica, sans-serif;">The same can be easily done with other components based on MDX query.</span><br /><span style="font-family: arial, helvetica, sans-serif;"><br /></span><br /><div style="margin: 0px; outline: none; padding: 0px;">
<span style="color: orange; font-family: arial, helvetica, sans-serif;">Using BTable Filter Panel From External Applications </span></div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">Sometimes it happens that in your custom application you need to work with dimension members selections (for example for profiling purposes). You can do it working directly on database but </span><span style="font-family: arial, helvetica, sans-serif;">I found it very useful to create one way to do it through BTable Filters Panel. Basically you have the opportunity to invoke BTable passing an endpoint as parameter. When the user saves filters selections the endpoint is launched.</span><br /><span style="font-family: arial, helvetica, sans-serif;">If you are curious about this, you can use comments to this post and I will do my best to explain it in details in another post.</span></div>
</div>
<span style="color: orange; font-family: arial, helvetica, sans-serif;"><br /></span><span style="color: orange; font-family: arial, helvetica, sans-serif;">Filter On Dimension Members </span></div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">When the user selects one dimension inside Filter Panel the dimension member showed are filtered based on filter selections made for other dimensions. This is the default behaviour but can be optionally changed by users. </span><br /><span style="font-family: arial, helvetica, sans-serif;"><br /></span><br /><div style="margin: 0px; outline: none; padding: 0px;">
<span style="color: orange; font-family: arial, helvetica, sans-serif;">Show Toolbar Option </span></div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">Now it is possible to show one toolbar with most common actions on top of BTable. Toolbar is active by default when you start from BTable Analyzer and viceversa for CDE dashboard. </span></div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">Users can toggle the toolbar visibility.</span></div>
<span style="font-family: arial, helvetica, sans-serif;"><br /></span><br /><div style="margin: 0px; outline: none; padding: 0px;">
<span style="color: orange; font-family: arial, helvetica, sans-serif;">History </span></div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">Since its first version BTable has the command Reset to reload the initial state. Now we also added the Back button in the toolbar that gives the opportunity to move BTable to previous states. </span></div>
</div>
</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;"><br /></span></div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<div style="margin: 0px; outline: none; padding: 0px;">
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="color: orange; font-family: arial, helvetica, sans-serif;">Show Zeros Option</span></div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">It is common in OLAP/MDX world to deal with NOT NULL option but it happens frequently that measures fields inside facts tables contain zeros values.</span><br /><span style="font-family: arial, helvetica, sans-serif;">This option, active by default, deletes rows and columns when all values are nulls or zeros.</span></div>
<span style="color: orange; font-family: arial, helvetica, sans-serif;"><br /></span><span style="color: orange; font-family: arial, helvetica, sans-serif;">Performance </span></div>
<div style="margin: 0px; outline: none; padding: 0px;">
<span style="font-family: arial, helvetica, sans-serif;">We made some improvements in order to speedup BTable rendering. I tested I'm able to list more then 300,000 rows in a reasonable amount of time. </span></div>
</div>
<div style="background-color: white; color: #333333; font-family: "Helvetica Neue Light", HelveticaNeue-Light, "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; margin: 0px; outline: none; padding: 0px; text-align: justify;">
<span style="font-family: arial, helvetica, sans-serif;"><br /></span><span style="font-family: arial, helvetica, sans-serif;">New posts with further details will follow.</span><br /><span style="font-family: arial, helvetica, sans-serif;"><br /></span><span style="font-family: arial, helvetica, sans-serif;">Enjoy!!</span></div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com3tag:blogger.com,1999:blog-6644329693530300467.post-75412900290431886232017-01-20T17:58:00.000+00:002017-01-20T17:58:03.412+00:00From 0 to a full blown Pentaho 7 spectacular dashboard in 60m<h3>
CBF2 is awesome? Hell yeah!</h3>
<div>
I've recently been blogging about <a href="http://pedroalves-bi.blogspot.pt/2016/04/cbf2-ultimate-collaboration-and-deploy.html">CBF2</a> and talking about how great it is. But I admit that even just by looking at the blog post some people may not take it seriously assuming it's too complex. It's not. </div>
<div>
<br /></div>
<h3>
What you'll get - in less than one hour</h3>
<div>
Today I did a demo on a topic that I'm extremely passioned about, horology. With the help of <a href="https://twitter.com/miguelacleite">Miguel Leite</a>, one of our <a href="http://www.pentaho.com/service/custom-dashboards">UX wizards here</a>, we did a one day push to build this project (bidder beware, this was a looooong day....). </div>
<div>
<br /></div>
<div>
The result? Absolutely spectacular, completely worth the effort: </div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghBa2YIDwsL3Az0v9RteJrk5_JQ0T74wdQB9yIwmpaxju6BZLQOVl54hh_UE5LRSOG60iCL-8OVOLqnII0SH4YH3CDqcDiwJhbvKEpzmwOjHREZnYs275P0QC_9Hum73MuuWVEAVRQa7A/s1600/dashboard.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="360" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEghBa2YIDwsL3Az0v9RteJrk5_JQ0T74wdQB9yIwmpaxju6BZLQOVl54hh_UE5LRSOG60iCL-8OVOLqnII0SH4YH3CDqcDiwJhbvKEpzmwOjHREZnYs275P0QC_9Hum73MuuWVEAVRQa7A/s400/dashboard.png" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
And all this fueled by the amazingly powerful dataservices + annotations:</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_PemF-VVjY8_miu_TMViwFh4uk0EHmNwLnCUxl6DMrao4nTKzZp3iOkwsnUgl3jUuk3anqL4tHufcIoTt81ps-mbfuik1VG0C8XQumzbO8ouqKdiwQ8bw92LG3MbfXVF_dfCMuxu5MrI/s1600/pdi.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="261" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_PemF-VVjY8_miu_TMViwFh4uk0EHmNwLnCUxl6DMrao4nTKzZp3iOkwsnUgl3jUuk3anqL4tHufcIoTt81ps-mbfuik1VG0C8XQumzbO8ouqKdiwQ8bw92LG3MbfXVF_dfCMuxu5MrI/s400/pdi.png" width="400" /></a></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
You can get this in even less than one hour; you know, it's just that most of the time is downloading stuff, and I kept getting distracted and forget to go back to what I was doing. I'm absolutely sure you can do it in much less!</div>
<div>
<br /></div>
<div>
So, let's go!</div>
<div>
<br /></div>
<h3>
Pre requisites</h3>
<div>
Here's what you need:</div>
<div>
<ul>
<li>Any operating system, and a machine with at least 8gb</li>
<li>Docker configured with at least 4gb on it (get it <a href="https://www.docker.com/products/overview">from here</a>)</li>
<li>Git (or any UI for git)</li>
<li>Not being afraid to launch a terminal window...</li>
</ul>
</div>
<div>
<br /></div>
<div>
C'mon, it's not asking much, is it?</div>
<h3>
Getting it all working in just 6 steps</h3>
<h4>
1. Create a directory for pentaho and CBF</h4>
<div>
Create a directory called pentaho, open a terminal there and clone CBF2</div>
<div>
<br /></div>
<blockquote class="tr_bq">
$ git clone https://github.com/webdetails/cbf2.git</blockquote>
<div>
<br /></div>
<div>
You should have all the directory structure <a href="http://pedroalves-bi.blogspot.pt/2016/04/cbf2-ultimate-collaboration-and-deploy.html">as described in the CBF2 blog post</a>. </div>
<div>
<br /></div>
<h4>
2. Download Pentaho 7.0</h4>
<div>
Under the software directory, create another folder, called 7.0.0.0-25 (I like to use the version / build number) and put pentaho there, CE or EE:</div>
<div>
<ul>
<li>Get CE <a href="https://sourceforge.net/projects/pentaho/files/Business%20Intelligence%20Server/7.0/">from sourceforge</a>: <i>pentaho-server-ce-7.0.0.0-25.zip</i></li>
<li>Get EE <a href="https://support.pentaho.com/hc/en-us/articles/216976286-Pentaho-7-0-GA-Release">from the Pentaho support portal</a> (customers only): <i>paz-plugin-ee-7.0.0.0-25-dist.zip, pdd-plugin-ee-7.0.0.0-25-dist.zip, pentaho-server-ee-7.0.0.0-25-dist.zip, pir-plugin-ee-7.0.0.0-25-dist.zip . </i>If you download patches for 7.0.0.0, they will be automatically applied. In this case you also need to put your license files under the <i>cbf2/licenses/</i> folder.</li>
</ul>
<div>
<br /></div>
</div>
<h4>
3. Get the horlogery-demo project</h4>
<div>
Clone the <a href="https://github.com/pmalves/horlogery-demo">horlogery-demo project</a> under the cbf2/projects directory:</div>
<div>
<br /></div>
<blockquote class="tr_bq">
$ git clone https://github.com/pmalves/horlogery-demo.git</blockquote>
<br />
<h4>
4. Do the CBF2 magic</h4>
<div>
Under the <i>cbf2/</i> folder you have the <i>cbf2.sh</i> magic script, built by pink unicorns. Go to that dir and...</div>
<div>
<ol>
<li>Execute cbf2 and press <i>[A]</i> to add a new image and select the server you downloaded. If you're using EE you'll need to accept the license agreement. A new image should be available</li>
<li>Execute cbf2 and press <i>[C]</i> to create a new project. Select the <i>horlogery-demo</i> project and the image created previously.</li>
<li>There's no 3</li>
</ol>
<div>
<br /></div>
</div>
<h4>
5. Start using it!</h4>
<div>
If everything went as expected, you should be seeing something like this:</div>
<div>
<br /></div>
<span style="font-family: Courier New, Courier, monospace;"><span style="font-size: xx-small;">pedro@orion:~/tex/pentaho/cbf2/projects/horlogery-demo (master *) $ cbf2<br /><br />Core Images available:<br />----------------------<br /> [0] baserver-ce-7.0.0.0-25<br /> [1] baserver-ee-7.0.0.0-25<br /><br />Core containers available:<br />--------------------------<br /><br />Project images available:<br />-------------------------<br /> [2] pdu-horlogery-demo-baserver-ce-7.0.0.0-25<br /> [3] pdu-horlogery-demo-baserver-ee-7.0.0.0-25<br /><br />Project containers available:<br />-----------------------------<br /><br />> Select an entry number, [A] to add new image or [C] to create new project:</span></span><div>
<br /></div>
<div>
<br /></div>
<div>
Select the project you want, press <i>[L]</i> to launch it and it will soon be available for you to start exploring!</div>
<div>
<br /></div>
<div>
<ul>
<li>Pentaho user console should be available at <a href="http://127.0.0.1:8080/">http://127.0.0.1:8080/</a></li>
<li>That great Ctools dashboard is avaiable at <a href="http://127.0.0.1:8081/pentaho/api/repos/:public:horlogery:Horlogery.wcdf/generatedContent">http://127.0.0.1:8081/pentaho/api/repos/:public:horlogery:Horlogery.wcdf/generatedContent</a></li>
</ul>
</div>
<div>
(Note that depending on the operating system, the docker IP may not be 127.0.0.1 though, I can't help there)</div>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrXU_XGbvUY6WrKDgsFOYWbMLyFyVTUJh_wxshoE8y1HweSy1-dT1ZV1aaqLpBe1EXrwxeR5vFukkDZmmBx_x-zTUcR_TuQiN78LQdzcA9SePjACa70x-BBTz9FX5g8iGjPGhft1dkg5s/s1600/puc.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="200" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrXU_XGbvUY6WrKDgsFOYWbMLyFyVTUJh_wxshoE8y1HweSy1-dT1ZV1aaqLpBe1EXrwxeR5vFukkDZmmBx_x-zTUcR_TuQiN78LQdzcA9SePjACa70x-BBTz9FX5g8iGjPGhft1dkg5s/s400/puc.png" width="400" /></a></div>
<div>
<br /></div>
<h4>
6. Next steps? </h4>
<div>
<br /></div>
<div>
From this point on it's you writing your own project and success story! And I'm going to get some sleep, since I had nearly none last night!! :p</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Have fun!</div>
<div>
<br /></div>
<div>
-pedro</div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com5tag:blogger.com,1999:blog-6644329693530300467.post-46239789931446526472017-01-10T15:54:00.000+00:002017-01-10T15:54:01.730+00:00Doing GeoLocation in PDI - Pentaho Data Integration (Kettle)<br />
<h3>
Geo Location</h3>
<br />
Geo location is something we often need in ETL work. And while we had a step that worked in PDI 5.x and earlier releases, we just noticed it's not currently working.<br />
<br />
Until this morning, that is :p<br />
<br />
I just forked <a href="https://mobile.twitter.com/mattcasters">Matt's</a> <a href="https://github.com/mattcasters/MaxMindGeoIPLookup">initial project</a> and <a href="http://pedroalves-bi.blogspot.pt/2015/10/pentaho-60-and-plugins-upgrade-from-vfs.html">applied the relevant changes</a> to make it compatible with Pentaho 6+<br />
<br />
<h3>
The basics</h3>
<div>
<br /></div>
<div>
Well, easy to understand... We have an IP address, we want to know where it comes from!</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjNgc3xsrZnBlMT3TZ00Mq3ZmxPf_tsav8m04p0I2yjmPxf8YFF6H7DoWe3d_uIQpeM6Q3BiuTJ5wsDM6MtyAB0YrAU_nrcgWre5ioeTZy_I6hXTCnHzPNuZEK6HCRFcT6eNF1Pjo7aIM/s1600/trans.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhjNgc3xsrZnBlMT3TZ00Mq3ZmxPf_tsav8m04p0I2yjmPxf8YFF6H7DoWe3d_uIQpeM6Q3BiuTJ5wsDM6MtyAB0YrAU_nrcgWre5ioeTZy_I6hXTCnHzPNuZEK6HCRFcT6eNF1Pjo7aIM/s400/trans.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Geolocation transformation - Let me see if it finds out where I am...</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
Once I execute this, I get the following result:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjn4MVSqyy93MLmHYKeOpXBo1LMyfhZ4ioBdSVX0GgdQiiXpKkRDLdO1lneMd4kYymAcBxW2198Bnly4D0FlT8MMutXUT0zqXdL_-jPuGJ-CeK9qYuJ6Hj31BsEtVw_iBPeJSUUgaytDlU/s1600/det.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjn4MVSqyy93MLmHYKeOpXBo1LMyfhZ4ioBdSVX0GgdQiiXpKkRDLdO1lneMd4kYymAcBxW2198Bnly4D0FlT8MMutXUT0zqXdL_-jPuGJ-CeK9qYuJ6Hj31BsEtVw_iBPeJSUUgaytDlU/s400/det.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Yep, this is where I am...</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
I am indeed in Porto Salvo, Portugal, so this is right. Can't get any easier than this!</div>
<div>
<br /></div>
<div>
<br /></div>
<h3>
Making it work</h3>
<div>
So, how to make this work? First, you have to get the plugin from the PDI marketplace</div>
<div>
<br /></div>
<div>
This plugin is available through the marketplace. Just go ahead and install it.</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjN9b9u6WDooJCSyhdTgqMZaL-4YPLmiVOKsrC5SKMYkBWQYVKJu4YwjKCohbbQ384n6kXQo7mr7bV0yEZWnGB1OqHlu4PN5z8_sRdhX6byGwVkRZxjr3T4ao7GIGp_L-CNTdrmrrujJ24/s1600/marketplace.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="273" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjN9b9u6WDooJCSyhdTgqMZaL-4YPLmiVOKsrC5SKMYkBWQYVKJu4YwjKCohbbQ384n6kXQo7mr7bV0yEZWnGB1OqHlu4PN5z8_sRdhX6byGwVkRZxjr3T4ao7GIGp_L-CNTdrmrrujJ24/s400/marketplace.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>PDI Marketplace - Get your goodies from here</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
After installing it and restarting PDI, you'll see the GeoIP Lookup step in the lookup folder. Configuring it is straightforward: You point to the stream field containing the IP address, point to the IP database files and specify what fields you want back:</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhPETKrLZmR_eJWhwVM7hc8iW7njLMKvqgXvVr2G8n2Gf8MV_GQEJXia-D9uKU0TBa-GHR0THmtsqbLOmALkq7E5yBDEdL_zt673j3vqSgBQqSXuErdNBTXEcjnN8xToDz9QgVFDIgiyk/s1600/stepConfig.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="180" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjhPETKrLZmR_eJWhwVM7hc8iW7njLMKvqgXvVr2G8n2Gf8MV_GQEJXia-D9uKU0TBa-GHR0THmtsqbLOmALkq7E5yBDEdL_zt673j3vqSgBQqSXuErdNBTXEcjnN8xToDz9QgVFDIgiyk/s400/stepConfig.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Configuring the step</i></td></tr>
</tbody></table>
<div>
<br /></div>
<h3>
Getting the IP Database files</h3>
<div>
You need to get the files from <a href="https://www.maxmind.com/en/home">MaxMind</a>, and from my experience these guys do a great job here. They have some great commercial offerings but also a <a href="https://dev.maxmind.com/geoip/legacy/geolite/">GeoLite database for country and city location. You can get them from here</a></div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeeELmlHPYaplWN6hniKQBFKPCjUXss0-iwLQmqcVmhPFxwrktJh9RhBjZAs3Wf6uye4rYObPPQVV1GKIz-nA8ad0HWeNu-9FMQp_48k-YAwZwwpH21lckNu5A6daPftdqJ7gjsMplPWw/s1600/GeoLite.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="352" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjeeELmlHPYaplWN6hniKQBFKPCjUXss0-iwLQmqcVmhPFxwrktJh9RhBjZAs3Wf6uye4rYObPPQVV1GKIz-nA8ad0HWeNu-9FMQp_48k-YAwZwwpH21lckNu5A6daPftdqJ7gjsMplPWw/s400/GeoLite.png" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>Getting the GeoIP data files</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
And you should be done! This even works great in a map reduce job</div>
<div>
<br /></div>
<div>
<br /></div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com10tag:blogger.com,1999:blog-6644329693530300467.post-78311317664663630932017-01-09T18:49:00.003+00:002017-01-09T18:49:33.090+00:00CBF2 now supporting multiple instances running<div>
<br /></div>
<div>
<br /></div>
<h3>
The scenario</h3>
<div>
<br /></div>
Last year I announced <a href="http://pedroalves-bi.blogspot.pt/2016/04/cbf2-ultimate-collaboration-and-deploy.html">CBF2</a>, the biggest, best, coolest way to manage Pentaho projects, available on <a href="https://github.com/webdetails/cbf2">github</a>. In case you don't recall, it relies on <a href="https://www.docker.com/">docker</a> to manage the images. Just read about it - really awesome<br />
<div>
<br /></div>
<div>
Since then, we've been using it <i>a lot</i> here. Really helps managing different projects and environments, and it's been put up to test in multiple real world scenarios.</div>
<div>
<br /></div>
<div>
One of the limitations of CBF2 is that it's limited to running one project per machine - since it exposes the ports on the host machine. </div>
<div>
<br /></div>
<div>
Another immediate consequence is that we can't have a local tomcat running since we'd get port conflicts.</div>
<div>
<br /></div>
<h3>
The need</h3>
<div>
However, sometimes it's useful to have running containers side by side. In this case, we wouldn't be able to run these two projects at the same time:</div>
<div>
<br /></div>
<div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1hpMLYjvekhm-5LfUKM75c0GgHL4L4ifR8U3xRrD862F1oYEH2dOVVsKD9_Q5xB33ovKxdvckyrKXtf1ZvJVocKAH5gc5bSV0LO_vKJnrTl4rTvCt91Nx40tjcOF8B5AP50iERjqcQis/s1600/img1.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="393" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh1hpMLYjvekhm-5LfUKM75c0GgHL4L4ifR8U3xRrD862F1oYEH2dOVVsKD9_Q5xB33ovKxdvckyrKXtf1ZvJVocKAH5gc5bSV0LO_vKJnrTl4rTvCt91Nx40tjcOF8B5AP50iERjqcQis/s640/img1.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>So I guess I have to run these two projects one at a time?</i></td></tr>
</tbody></table>
<br />
If you tried to run these two it would complain about conflicting ports and the likes of it.<br />
<br />
<h3>
The solution</h3>
<br />
Turns out <a href="http://kleysonrios.blogspot.pt/">Kleyson Rios</a> was less clumsy than I am - so he implemented this feature in CBF2: The ability to have containers running side by side by automagically detecting used ports and just moving on to the next one.<br />
<br />
In here you see that both containers successfully ran:<br />
<br />
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKu90izOm0y1Sk27BAGbmnwenyEKRb4k2a2BQ2OYn3nc-H9WdB9zRBY3b32VWUbmv5_gzMiaXVc-9AVM8uzW8furqRDVFwuUwKmCQGeGiJgYVz7TwZZIQUcp1jeXA1DnHTK2uxtiAMQCE/s1600/img2.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="394" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjKu90izOm0y1Sk27BAGbmnwenyEKRb4k2a2BQ2OYn3nc-H9WdB9zRBY3b32VWUbmv5_gzMiaXVc-9AVM8uzW8furqRDVFwuUwKmCQGeGiJgYVz7TwZZIQUcp1jeXA1DnHTK2uxtiAMQCE/s640/img2.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>2 projects running side by side</i></td></tr>
</tbody></table>
<br />
<h3>
The result</h3>
</div>
<div>
<br /></div>
<div>
The end result? Pretty cool, I have to admit! I now can run and test different versions side by side on my machine just by using the correct port :)</div>
<div>
<br /></div>
<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxar4LgmEXh0REEoSJvJKkZgzOYcBFBn-UtMCKeITEJpcYpoauI3CnT5dmBLewk5dDAY_4Zc6v1X64HFBMqM9doSipMPLDS9Lh68Z2tw8VQ6lYJDEWG-qpDSahxZXLmudlYgLuglYVVC8/s1600/puc.png" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="270" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgxar4LgmEXh0REEoSJvJKkZgzOYcBFBn-UtMCKeITEJpcYpoauI3CnT5dmBLewk5dDAY_4Zc6v1X64HFBMqM9doSipMPLDS9Lh68Z2tw8VQ6lYJDEWG-qpDSahxZXLmudlYgLuglYVVC8/s640/puc.png" width="640" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>2 different versions, 5.4 and 7.1, running on my machine with 2 simple commands</i></td></tr>
</tbody></table>
<div>
<br /></div>
<div>
<br /></div>
<div>
This improvement is already committed, so simply <a href="https://github.com/webdetails/cbf2">pull the latest version</a> and you're ready to go.</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
Thanks Kleyson! :)</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
-pedro</div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com2tag:blogger.com,1999:blog-6644329693530300467.post-88546892659388803202016-12-20T19:32:00.000+00:002016-12-20T19:32:11.258+00:00New WEKA releases: 3.6.15, 3.8.1 and 3.9.1The Weka team is on fire. New releases available for download from the Weka homepage:<br />
<br />
<h4>
Weka 3.8.1 - stable version. </h4>
It is available as ZIP, with Win32
installer, Win32 installer incl. JRE 1.8.0_112, Win64 installer, Win64
installer incl. 64 bit JRE 1.8.0_112 and Mac OS X application with
Oracle 64 bit JRE 1.8.0_112.<br />
<h4>
Weka 3.9.1 - development version</h4>
It is available as ZIP, with
Win32 installer, Win32 installer incl. JRE 1.8.0_112, Win64 installer,
Win64 installer incl. 64 bit JRE 1.8.0_112 and Mac OS X application with
Oracle 64 bit JRE 1.8.0_112.<br />
<br />
<h4>
Weka 3.6.15 - stable book 3rd edition version</h4>
<br />
It is available as
ZIP, with Win32 installer, Win32 installer incl. JRE 1.8.0_112, Win64
installer, Win64 installer incl. 64 bit JRE 1.8.0_112 and Mac OS X
application with Oracle 64 bit JRE 1.8.0_112.<br />
<br />
Stable 3.8 receives bug fixes and new features that do not include
breaking API changes and maintain serialized model compatibility. 3.9
(development) receives bug fixes and new features that might include
breaking API changes and/or render models serialized using earlier
versions incompatible.<br />
<br />
NOTE: 3.6.15 is the final release of stable-3-6.<br />
<br />
Weka homepage:<br />
<a data-saferedirecturl="https://www.google.com/url?hl=en&q=http://www.cs.waikato.ac.nz/~ml/weka/&source=gmail&ust=1482343528220000&usg=AFQjCNH4xhnStt6gkyfARX1x9CPhniiSZg" href="http://www.cs.waikato.ac.nz/~ml/weka/" rel="noreferrer" target="_blank">http://www.cs.waikato.ac.nz/~<wbr></wbr>ml/weka/</a><br />
<br />
Pentaho data mining community documentation:<br />
<a data-saferedirecturl="https://www.google.com/url?hl=en&q=http://wiki.pentaho.com/display/Pentaho%2BData%2BMining%2BCommunity%2BDocumentation&source=gmail&ust=1482343528220000&usg=AFQjCNFBUNCbC_6fX4mM3Mit3RsEVJcBQg" href="http://wiki.pentaho.com/display/Pentaho+Data+Mining+Community+Documentation" rel="noreferrer" target="_blank">http://wiki.pentaho.com/<wbr></wbr>display/Pentaho+Data+Mining+<wbr></wbr>Community+Documentation</a><br />
<br />
Packages for Weka>=3.7.2 can be browsed online at:<br />
<a data-saferedirecturl="https://www.google.com/url?hl=en&q=http://weka.sourceforge.net/packageMetaData/&source=gmail&ust=1482343528220000&usg=AFQjCNGzgYe3NQuN6ySqRpzaLGVyq9og2g" href="http://weka.sourceforge.net/packageMetaData/" rel="noreferrer" target="_blank">http://weka.sourceforge.net/<wbr></wbr>packageMetaData/</a><br />
<br />
<br />
<h3>
What's new in 3.8.1/3.9.1?</h3>
<br />
Some highlights<br />
---------------<br />
<br />
In core weka:<br />
<br /><ul>
<li>Package manager now handles redirects generated by SourceForge</li>
<li>Package manager now employs a new class loading mechanism that
attempts to avoid third-party library clashes by isolating the
third-party libraries in each package</li>
<li>new RelationNameModifier, SendToPerspective, WriteWekaLog, Job,
StorePropertiesInEnvironment, SetPropertiesFromEnvironment,
WriteDataToResult and GetDataFromResult steps in Knowledge Flow</li>
<li>RandomForest now has an option for computing the mean impurity decrease variable importance scores</li>
<li>JRip now prunes redundant numeric attribute-value tests from rules</li>
<li>Knowledge Flow now offers an additional executor service that uses
a single worker thread; steps can, if necessary, declare
programmatically that they should run in the single-threaded executor.</li>
<li>GUIs with result lists now support multi-entry delete</li>
<li>GUIs now support copying/pasting of array configurations to/from the clipboard</li>
</ul>
<br />
In packages:<br /><ul>
<li>Multi-class FLDA in the discriminantAnalysis package</li>
<li>New implementations in the ensemblesOfNestedDichotomies package</li>
<li>distributedWekaBase now includes the latest version of Ted
Dunning's t-digest quantile estimator, bringing a factor of 4 speedup
over the old implementation</li>
<li>New streamingUnivariateStats package</li>
<li>RPlugin package updated to support the latest version of MLR</li>
<li>New wekaDeepLearning4j package - provides a MLP classifier built
using the DL4J library. Can work with either CPU-based or GPU-based
native libraries</li>
<li>New logarithmicErrorMetrics package</li>
<li>New RankCorrelation package, courtesy of Quan Sun. Provides rank
correlation metrics, Kendall tau and Spearman rho, for evaluating
regression schemes</li>
<li>New AffectiveTweets package, courtesy of Felipe Bravom. Provides text filters for sentiment analysis of tweets</li>
<li>New AnalogicalModeling package, courtesy of Nathan Glenn. Provides an exemplar-based approach to modeling</li>
<li>New MultiObjectiveEvolutionaryFuzz<wbr></wbr>yClassifier package, courtesy of Carlos Martinez Cortes. Provides a fuzzy rule-based classifier</li>
<li>New MultiObjectiveEvolutionarySear<wbr></wbr>ch package, courtesy of
Carlos Martinez Cortes. Provides a search method that uses the ENORA
multi-objective evolutionary algorithm</li>
</ul>
<br />
As usual, for a complete list of changes refer to the changelogs.<br />
<br />Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com2tag:blogger.com,1999:blog-6644329693530300467.post-88157661630047384872016-11-16T15:06:00.002+00:002016-11-16T15:06:31.790+00:00Oh. btw, Pentaho 7.0 is out!<table align="center" cellpadding="0" cellspacing="0" class="tr-caption-container" style="margin-left: auto; margin-right: auto; text-align: center;"><tbody>
<tr><td style="text-align: center;"><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCBZA0J_aLqFGfbEKSEmD-WAe4cRX4ghRFgw1jTf-YldM720y0cw7fg3mg_bPoNJ2gXOq7xn2xG4nf_CpO-siqazCAIccdPjQV7cGc4QySp4oV43sj1wBGKO43AzvF_csn1cLA3wa2dt8/s1600/pcm16.jpg" imageanchor="1" style="margin-left: auto; margin-right: auto;"><img border="0" height="113" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjCBZA0J_aLqFGfbEKSEmD-WAe4cRX4ghRFgw1jTf-YldM720y0cw7fg3mg_bPoNJ2gXOq7xn2xG4nf_CpO-siqazCAIccdPjQV7cGc4QySp4oV43sj1wBGKO43AzvF_csn1cLA3wa2dt8/s400/pcm16.jpg" width="400" /></a></td></tr>
<tr><td class="tr-caption" style="text-align: center;"><i>#PCM16 group photo - once again I was outside having a drink</i></td></tr>
</tbody></table>
<br />
<br />
I almost forgot! Pentaho 7.0 is out! We chose the amazing <a href="http://www.know.bi/blog/article/announcing-pcm16">Pentaho Community Meeting</a> (<a href="https://twitter.com/search?src=typd&q=%23pcm16">#PCM16</a>) as the release date (talk about commitment to the community ;) ), so just go get them at the usual places, both <a href="http://www.pentaho.com/testdrive">Enterprise Edition</a> as well as <a href="http://community.pentaho.com/">Community Edition</a><br />
<br />
<br />
-pedroPedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com1tag:blogger.com,1999:blog-6644329693530300467.post-28515265268174956532016-11-02T12:44:00.000+00:002016-11-02T12:44:16.877+00:00PCM16 - Pentaho Community Meeting, coming up Nov 11-132 weeks to go. For the 9th year in a row, this fantastic event will take place in less than 2 weeks in Antwerp. See you soon!!<br />
<br />
<br />
<br />
Now, let me shamelessly <a href="http://www.know.bi/blog/article/announcing-pcm16">copy the blog post</a> from <a href="https://twitter.com/bartmaer">Bart Maertens</a>, the organizer of the event:<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.know.bi/images/uploads/global/Knowbi_520x346.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.know.bi/images/uploads/global/Knowbi_520x346.jpg" height="212" width="320" /></a></div>
<br />
<div class="w-col w-col-6 w-col-stack">
<div class="product-detail-info blog-detail" data-ix="from-left" style="opacity: 1; transform: translateX(0px) translateY(0px); transition: opacity 500ms ease 0s, transform 500ms ease 0s;">
<h1>
</h1>
<h1>
Announcing #PCM16, Antwerp, Belgium!!</h1>
<br />
<strong>Register now!<br />
Friday, Nov 11th:
<a href="https://www.eventbrite.com/e/pcm16-hackathon-tickets-26473080697">PCM16 Hackathon<br />
</a>Saturday, Nov 12: <a href="https://www.eventbrite.com/e/pcm16-tickets-26473010487">PCM16</a></strong><br />
<strong>Submit your talk proposal!<br />
</strong><br />
<strong>Use case room: <a href="mailto:pcm16_biz@know.bi">pcm16_biz@know.bi<br />
</a></strong><strong>Tech room: <a href="mailto:pcm16_tech@know.bi">pcm16_tech@know.bi</a></strong><br />
<strong>AGENDA</strong>: What's cooking a PCM16? Have a look:<br />
<strong>ANTWERP: </strong>How to get there and things to do in<br />
After the 2015 edition in London, the eighth yearly Pentaho Community
Meeting will be back where it was in 2014: Antwerp, Belgium. The dates
for PCM16 will be Friday, November 11th and Saturday, November 12th. The
venue for this edition will be the gorgeous medieval hospital and
monastery
<a href="http://www.elzenveld.be/">Elzenveld</a>. The views may not be as spectacular as they were in the 2013 Sintra edition, but the location sure will be fine!<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://www.know.bi/images/uploads/global/1_slider_tuinelzenveldslider.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="http://www.know.bi/images/uploads/global/1_slider_tuinelzenveldslider.jpg" height="213" width="320" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<br />
<br />
As was the case in the 2014 and 2015 editions, we’ll have a hackathon
(followed by drinks) on Friday evening and two presentation rooms
(business and technical) on Saturday.<br />
As has been the trend in the latest PCM editions, we aim to make this
the European Pentaho event of the year for both Enterprise and
Community Edition users. After all, no matter which version you use,
we’re all just a community of Pentaho users.<br />
The event is free of charge because of sponsorships by know.bi and
Pentaho, there will only be a small charge (€10) for lunch which you
will kindly be asked to pay in cash when registering on Saturday.<br />
In return for a weekend of your time, you’ll enjoy a couple of days
of being submerged in everything Pentaho, (Big) Data, Data Science and
the excitement of talking to and working with the community involved in
all of this.<br />
Registrations are open now, register on our eventbrite pages for the hackathon and PCM:<br />
Friday, Nov 11th:
<a href="https://www.eventbrite.com/e/pcm16-hackathon-tickets-26473080697">PCM16 Hackathon</a><br />
Saturday, Nov 12:
<a href="https://www.eventbrite.com/e/pcm16-tickets-26473010487">PCM16</a><br />
<h1>
Friday, November 11th</h1>
On the evening of Friday, November 11th, we’ll be hosting a
hackathon. People will have to travel to Antwerp, therefore we won’t be
able to start early (8PM-ish) and have a hackathon for hours on end.
However, as has been shown in the previous years, a couple of hours
suffice to build and present impressive solutions with PDI, Mondrian or
CTools.<br />
As tradition has it, beer is an important part of a pre-PCM Friday
evening, and there’s nowhere better to go for beers than Belgium! There
are quite a number of pubs in the vicinity of the venue:
<a href="https://www.facebook.com/pages/K-Zeppos/150039021725140">‘K. Zeppos’</a>, named after the -at least in Belgium- world famous sixties TV series <a href="https://www.youtube.com/watch?v=CNUaGF5VhpE">‘Kapitein Zeppos</a>’ and <a href="https://www.facebook.com/pallietermechelseplein">‘Pallieter</a>’, named after a 1916 <a href="https://nl.wikipedia.org/wiki/Pallieter_%28roman%29">novel</a> by Flemish writer <a href="https://nl.wikipedia.org/wiki/Felix_Timmermans">Felix Timmermans</a>, just to name a few.<br />
After a couple of hours of hacking, this is the perfect excuse to enjoy some of our famous
<a href="https://en.wikipedia.org/wiki/Beer_in_Belgium">Belgian beers</a>. Take it easy though, these are not Amstel or Heineken!<br />
<h1>
Saturday, November 12th</h1>
<h2>
The rooms</h2>
Traditionally, Saturday is what a true PCM is all about!<br />
Just like in the last editions, there will be two rooms: business and technical.<br />
The business room will be your goto place for use case presentations
where Pentaho customers and/or users explain what real life problems
they are solving with the Pentaho suite.<br />
Some of Pentaho’s biggest and most prestigious implementations will
be presented here, alongside smaller but not less interesting
implementations.<br />
The technical room is for more technical presentations. This is the
‘old school’ Community Meeting room, and your goto place to find out
what’s cooking within Pentaho and the Pentaho Community. Powerpoint used
to be forbidden, just like beaming code on the big screen was mandatory
in the earliest Pentaho Community Meetings, find out for yourself if
this still stands.<br />
<h2>
Call for Speakers</h2>
A list of speakers for both rooms is currently being compiled.<br />
If you’d like to present, please mail to
<a href="mailto:pcm16_biz@know.bi">pcm16_biz@know.bi</a> or <a href="mailto:pcm16_tech@know.bi">pcm16_tech@know.bi</a> with a short description of your presentation proposal and we’ll get back to you asap.<br />
<h1>
Sunday, November 13th</h1>
Again, not intending to change a winning team and sticking to tradition, we’ll have a social activity on the post-PCM Sunday.<br />
An agenda is still being compiled, but we’ll post regular updates here, so stay tuned for more.<br />
<h1>
How to get there</h1>
<strong>By Plane</strong><br />
When you're arriving in Belgium through
<a href="http://www.know.bi/customer-cases/case/the-brussels-airport-company" target="_blank">Brussels Airport</a>, there are <a href="http://www.brusselsairport.be/en/passngr/to_from_brussels_airport/train/" target="_blank">direct trains</a> from the airport to Antwerp (approximately 30 minutes).<br />
A (limited) number of cities have direct connections to
<a href="http://www.antwerp-airport.be/contentpage_en.php" target="_blank">Antwerp Airport</a>, which is just a 15 minute taxi ride from the city center.<br />
<strong>By Train</strong><br />
Antwerp has connections to several European cities, including a number of high speed connections. Check out the Belgian Rail
<a href="http://www.belgianrail.be/en/stations-and-train/search-a-station/6/antwerpen-centraal.aspx" target="_blank">website</a> for more details.<br />
<strong>By car</strong><br />
Follow your GPS to you hotel's address. Driving in Antwerp is fine, but traffic around the city can be challenging (
<a href="http://uk.businessinsider.com/cities-with-the-worst-traffic-in-europe-2016-3" target="_blank">slow</a>).<br />
<h1>
Agenda</h1>
<strong>Technical Room</strong><br />
<ul>
<li><a href="https://twitter.com/magicaltrout">Tom Barber</a>: Alternative Big Data Devops</li>
<li><a href="https://twitter.com/JensBleuel">Jens Bleuel</a>: What’s new in PDI 7.0</li>
<li><a href="https://twitter.com/mattcasters">Matt Casters</a>: PDI Unit Testing</li>
<li><a href="https://twitter.com/know_bi">know.bi</a>: Loading data to Neo4J using PDI </li>
<li><a href="https://twitter.com/ubiquisBI">UbiquisBI</a>: Data models for Hadoop: Kimball without updates</li>
<li><a href="https://twitter.com/Wael_Elrifai" target="_blank">Wael Elrifai</a>: TBD</li>
<li><a href="https://twitter.com/Pentaho" target="_blank">Pentaho</a> - WebDetails: TBD </li>
<li>Hiromu Hota ## smell of announcement ##</li>
</ul>
<strong>Use Cases</strong><br />
<ul>
<li><span pt="" sans-serif="" sans=""><a href="https://www.abnamroclearing.com/en/index.html">ABN AMRO Clearing</a> </span></li>
<li><a href="http://66controls.eu/en/home-en/">66 Controls</a></li>
<li><a href="http://www.know.bi/customer-cases/case/cipal">Cipal</a></li>
<li><a href="http://www.know.bi/customer-cases/case/the-brussels-airport-company" target="_blank">Brussels Airport</a></li>
</ul>
</div>
</div>
Pedro Alveshttp://www.blogger.com/profile/17239718993167083071noreply@blogger.com2