The SAP HANA Data Warehousing Foundation (DWF) option is a series of packaged tools for large-scale SAP HANA installations, which support data management and distribution within a SAP HANA landscape. With SAP HANA Data Warehousing Foundation you can achieve smart data distribution across complex landscapes, optimize the memory footprint of data in SAP HANA and streamline administration and development, thereby reducing TCO and supporting SAP HANA administrators and data warehouse designers.

In this blog I will focus on Data Lifecycle Manager (DLM).  Please note there are other tools as part of the DWF option that we will not look at during this blog. 

SAP HANA is the core platform in any SAP Big Data target architecture. One of the challenges you will face with any Big Data Architecture is managing tiered storage.   You need to make intelligent decisions on how to optimize and balance price and performance.

For the most part in the past you had to manually manage and develop your tiered data processes including: data relocation, monitoring, logging, scheduling, testing and harmonizing tiers.   I will show how DLM addresses these challenges.  This is not a training session.  The purpose is to give you a high-level understanding of the benefits of using DLM as part of your Big Data tiering/archiving strategy.  You should get a better understanding on how this can reduce cost and complexity while simplifying administration and development.

Below is a high level view on the DLM process:

Before we get started, let’s have a look at the table/data that we will be creating a data relocation profile for.  The table contains data that is the result of a computation over the Call Detail Records (CDRs) generated by the Telecom Italia cellular network over the city of Milano. CDRs log the user activity for billing purposes and network management. Below is simply the record count by country.

1 - Once you have DLM installed and configured you can access the xs application via

http(s)://<HOST>:<HANA XS PORT/sap/hdm/dlm/index.html

Here is a screen shot of the home page.  It’s a great place to see a quick snapshot of the last data relocation runs and node utilization forecast.

2 – The first thing to take care of will be to add a storage destination. Currently DLM supports 4 different storage destinations:

1. SAP IQ over SDA
2. HANA Dynamic Tiering Local
3. Spark SQL (DESTINATION)
4. Deletion Bin

Each of the different storage destinations has prerequisites.  To use HANA Dynamic tiering you need to have configured extended storage for HANA.  For Spark SQL you need the Spark Controller and Hadoop configured.

3 – Once you have your storage destination configured and activated you will be able to create Lifecycle Profiles.  The below profile is based on a source HANA table.  The destination will be spark.  All data is in the hot store(2459324).

4 – Once you have your profile created and source/targets identified you need to create a rule that DLM will use to tier the data.  As seen in the below screen shot I have defined the rule to move any record with COUNTRY_CODE = ‘40’ to cold storage.  The editor in real-time gives you information on the number of rows effected by the newly created rule.  In a production environment your rule would most likely be dynamic based on the date.

5 – Once your rule has been defined and saved DLM gives you the ability to “Simulate” the data relocation.

6 – Below you can see the simulation shows 100,740 records will move from Hot to Cold.

7 – If you are confident in the data relocation simulation you can move to scheduling the process.  For this blog I’m going to trigger the process manually.

8 -  The logs section provides an area to monitor the jobs progress in real-time as well as look at previously executed jobs and the associated steps.  Below you can see the detailed steps and associated status for the CDR job we manually executed.

9 – Now that the data has been relocated to cold storage we can take advantage of some generated objects that DLM has created to optimize access between the hot and cold data. Below you can see 2 new objects created by DLM.

                  1 – Virtual table

                  2 – Union view that combines both hot and cold data.

10 – Selecting from the Virtual table should only show the data that has been relocated to Hadoop.

11 – Selecting from the DLM generated view returns us the data located in both HANA and Hadoop.

Conclusion: As you can see managing your data across tiers becomes seamless and integrated with DLM.  You can take advantage of the DLM cockpit to create, mange and monitor all your data relocation jobs.  The generated objects should help your applications and analytics take advantage of all your enterprise data regardless of its temperature/age.

Next Steps: In part 2 of this blog series we will take a deeper look at the Data that was relocated to Hadoop and how we interact with it using VORA.

When installing SAP HANA, you have several choices to make:

• Use the graphical or console interface?
• What HANA core and option software to install?
• Add the worker host during installation or after the fact?

With so many choices, finding the steps you need for each task can be a challenge. Wouldn’t it be nice if the documentation could help you figure it out and guide you there?

Well now it can.  We’ve added interactive graphics to the installation tasks in the SAP HANA Dynamic Tiering Installation and Update Guide, to help you make choices and guide you through each task in the installation process.

At each stage in the installation process, we’ll show you where you are,

and what choices are available for the next step.

We’ll show you an overview of the whole process,

but won’t let you try a step until you’re ready for it.

When you click an available step,

we’ll take you to the relevant task.

When you complete that task, click your next available step,

To go right there.

Interactive graphics guide you through the installation process, helping you find the right tasks when you need them. Take a look at the SAP HANA Dynamic Tiering Installation and Update Guide on the Help Portal and give it a try.

Note:  Interactive graphics are not available in PDF output.

Introduction

I have a HANA XS application that can be deployed to separate tenant databases when my HANA system is setup as Multitenant Database Containers (MDC).  Naturally, I require HTTP access to those tenant databases and I was running into problems configuring this for both my SystemDB and my tenant databases.

I found the excellent post by Wenjun Zhou Exposing HTTP access to multitenant database containers in SAP HANA SPS09 and although the procedure is different for HANA SPS11, it pointed me in the right direction.  The documentation also has a section on this topic (Configure HTTP(S) Access to Multitenant Database Containers - SAP HANA Administration Guide - SAP Library) that was useful, but there were a few steps that needed extra attention to complete the process, which I'll describe in this blog post.

Note: I'll only discuss HTTP access here, but the procedure is similar to configure HTTPS access.

Assumptions

• The HANA databases in your system should already be set up for access via DSN alias hostnames.
• The Web Dispatcher is set for automatic configuration.
• You have already created one or more tenant databases in your HANA MDC system.

There are a few other prerequisites and the list is found in the documentation link above.

Accessing the XS Engine

By default, you can access the XS engine for the SystemDB by opening the URL http://<fqdn_hostname>:80<instance>, where "fqdn" is the fully qualified domain name (FQDN).  For example, if my HANA system is "mdc.internal.sap.com" and instance number is 00, then the XS engine URL for the SystemDB is http://mdc.internal.sap.com:8000.

Now, if you try to access the XS engine for one of your tenant databases (assuming you've set up its DSN alias hostname), you'll immediately get the following error:

503 Service not available

For example, if you created a tenant database called "TN1" with alias "mdc-tn1", then the XS engine for that database is http://mdc-tn1.internal.sap.com:8000 - however you'll get that 503 error when you try to open this URL.

Configure HTTP Access for the Tenant Database

What you must do now is specify the public URLs for all your tenant databases.  Here's how you do this in HANA Studio (example with SQL is below as well):

1. Connect to the SYSTEMDB (not the tenant database) as SYSTEM and open Administration.
2. Switch to the Configuration tab and expand xsengine.ini | public_urls.  Double-click on "public_urls" to invoke the Change Configuration Values dialog.
   
   
   
   
3. Click the Databases drop-down box, select the tenant database(s) you want to configure and click OK.
   
   
   
   
4. Provide the new URL using the alias hostname and click Save.
   
   
   
   

You are now able to XS engine for the tenant database (e.g. http://mdc-tn1.internal.sap.com:8000) without getting the 503 error.

If you want to use SQL instead of HANA Studio, you can execute the following statement in a SQL Console.  Don't forget to connect to the SYSTEMDB (not the tenant database!) as the SYSTEM user.

ALTER SYSTEM ALTER CONFIGURATION ('xsengine.ini', 'database', 'TN1')
SET ('public_urls', 'http_url') = 'http://mdc-tn1.internal.sap.com:8000'
WITH RECONFIGURE;

Re-Configure HTTP Access for the SystemDB

If you now try to access the SystemDB XS engine using the same URL as before (e.g. http://mdc.internal.sap.com:8000), you'll find that you'll get (oh no!):

503 Service not available

The documentation clearly states that the URL for the SystemDB changes once you've configured the URLs for the tenant databases.  Instead of using the FQDN, the SystemDB XS engine is available under http//<localhost>:80<instance>.  In our example, this means that the XS engine would now be found at http//mdc:8000.

What you need to do now is execute the following SQL statement to re-configure the SystemDB URL to use the alias host name:

ALTER SYSTEM ALTER CONFIGURATION ('nameserver.ini', 'system')
SET('public_urls', 'http_url') = 'http://mdc.internal.sap.com:8000'
WITH RECONFIGURE;

Note that if you try to make this change in HANA Studio by editing the xsengine.ini configuration file, the change will not take effect.

Voila!  You now have access to the XS engines for the SystemDB and tenant databases using the alias hostnames.

Final Thoughts

You can spend a lot of time trying to figure out how to enable HTTP access to your SystemDB and tenant databases.  Usually once you configure the tenant databases properly, you scratch your head determining why the SystemDB URL no longer works.  The procedure to fix this is quite straightforward, once you know where to look.  At a minimum, following these steps will allow you to open each tenant database's HANA Cockpit, as that's also a HANA XS application.

    I came across a number of text mining examples and all them where dealing with text information that is not from my area of expertise, for example, microbiology. Such examples are not catching your attention because you can not really tell if text mining is working based on your knowledge. I created SAP TechEd 2015 Session Catalogue data to play with HANA Text Mining. Any SAP professional is familiar with this text data and can make his opinion about HANA Text Mining.

    So here is my data. As you can see there are SESSION and TITLE information fields, CATEGORY field that classifies each document and DESCRIPTION field which contains text data for text mining (full-text index is built on this field).

I prepared a number of examples. Lets see text mining in action.

Find Similar Documents

For example, find session similar to DEV260 'Building Applications with ABAP Using Code Pushdown to the Database'

Find Relevant Terms

For example, find terms relevant to DEV260 'Building Applications with ABAP Using Code Pushdown to the Database' session

Find Related Terms

For example, find terms related to 'Fiori'

Find Relevant Documents

For example, find documents relevant for term 'Fiori'

Categorize Documents

For example, you have a new session for which you have to assign to a proper category. I took SAP TechEd 2014 DEV161 'SQLScript – Push Code Down into SAP HANA to Achieve Maximum Performance' session which belongs to 'Development and Extension Platform for SAP HANA and Cloud' category and classified it using 2015 SAP TechEd catalog documents. Lets see if the document will be correctly classified.

As you can see the document was classified correctly.

You can import attacted delivery unit into you HANA system (change extenstion to *.tgz first) and play with examples and data.

Once you delivery unit is imported you will have following objects in tm_demo package created

Note: described examples are in query.sql file

Excecute install.sql script to assign proper authorizations, fix data in session table data and create full text index for text mining.

Following catalog objects will be created

Note: for text mining function to work correctly you need to be on HANA SPS10

Would you like to understand how to optimize pricing and triple your profit in just a few minutes?

Of course you would! Keep reading to learn how one organization has built an R custom component that allows business users to access a powerful optimization algorithm quickly and easily.

Input Data
For this particular use case, the pricing department for our manufacturing company would like to know how to best price each of our products for next year. For this simple case, we’ll use 9 products, and include the product name (or SKU identifier), the current annual volume sold, current price of the product, and current profit margin.

Some of the products are currently unprofitable (as evidenced by the negative profit margin), so management would like to re-set prices for next year to ensure that:

1.) No product has a price increase or decrease of over 10%

2.) Any product that is currently unprofitable has a minimum price that is break-even for profit

3.) Overall, given the same volumes as this year, total revenue will increase by 2-4%

Visualizations

After importing the dataset we can build some basic visualizations to help understand our current product mix. A few examples are below, showing that Wallets, Necklaces, Hats, and Jackets account for the highest volumes of products sold, while Pants and Earrings are the highest priced (per unit). The stacked bar charts show how much revenue and profit each product accounts for—consistent with the profit margins listed in the original data, several products account for negative profit.

A sample example for XSOdata  with Navigation Property

Scenario of Purchase Order(PO) Header andItem: In a single service we can navigate to Item level from Header to Item with Association .

Lets start with very basic example of VBAK (PO Header), VBAP(PO Item).

Example Code in a .XSOdata file

service namespace "workshop.aisurya.XSOdata" {
"<YourSchema>"."VBAK" as "POHeader" with ("MANDT","VBELN","ERDAT") navigates ("ToPoItemAssctn" as "ItemRef");
"<YourSchema>"."VBAP" as "POItems" with ("MANDT","VBELN","POSNR","MATNR","ARKTX"); 
association "ToPoItemAssctn" principal "POHeader"("VBELN") 
multiplicity "1" dependent "POItems"("VBELN") multiplicity "*";
}

Below Image for reference

Activate the changes and Run As XS Service.

Lets check the Metadata of service.

https://<Host:Port>/workshop/aisurya/XSOdata/po.xsodata/$metadata

As highlighted in the image(ItemRef) is a navigation property from Header to Item level.

Lets test the service only for 2 records by providing query option as (top=2).

https://<Host:Port>/workshop/aisurya/XSOdata/po.xsodata/POHeader?$top=2&$format=json

Here ItemRef is the navigation property of corresponding PO Items with URI. We can explicitly open in new tab and get the result also .

Now lets check Expand option for Showing Header and Corresponding Items.

$Expand Feature to fetch Corresponding Item along with Header.

https://<Host:Port>/workshop/aisurya/XSOdata/po.xsodata/POHeader?$top=2&$format=json&$expand=ItemRef

POHeader (MANDT='800', VBELN='0000004970')

Now this service can be consumed in different UI ‘s  depending upon the requirement.

Introduction

I have a HANA XS application that can be deployed to separate tenant databases when my HANA system is setup as Multitenant Database Containers (MDC).  Naturally, I require HTTP access to those tenant databases and I was running into problems configuring this for both my SystemDB and my tenant databases.

I found the excellent post by Wenjun Zhou Exposing HTTP access to multitenant database containers in SAP HANA SPS09 and although the procedure is different for HANA SPS11, it pointed me in the right direction.  The documentation also has a section on this topic (Configure HTTP(S) Access to Multitenant Database Containers - SAP HANA Administration Guide - SAP Library) that was useful, but there were a few steps that needed extra attention to complete the process, which I'll describe in this blog post.

Note: I'll only discuss HTTP access here, but the procedure is similar to configure HTTPS access.

Assumptions

• The HANA databases in your system should already be set up for access via DSN alias hostnames.
• The Web Dispatcher is set for automatic configuration.
• You have already created one or more tenant databases in your HANA MDC system.

There are a few other prerequisites and the list is found in the documentation link above.

Accessing the XS Engine

By default, you can access the XS engine for the SystemDB by opening the URL http://<fqdn_hostname>:80<instance>, where "fqdn" is the fully qualified domain name (FQDN).  For example, if my HANA system is "mdc.internal.sap.com" and instance number is 00, then the XS engine URL for the SystemDB is http://mdc.internal.sap.com:8000.

Now, if you try to access the XS engine for one of your tenant databases (assuming you've set up its DSN alias hostname), you'll immediately get the following error:

503 Service not available

For example, if you created a tenant database called "TN1" with alias "mdc-tn1", then the XS engine for that database is http://mdc-tn1.internal.sap.com:8000 - however you'll get that 503 error when you try to open this URL.

Configure HTTP Access for the Tenant Database

What you must do now is specify the public URLs for all your tenant databases.  Here's how you do this in HANA Studio (example with SQL is below as well):

1. Connect to the SYSTEMDB (not the tenant database) as SYSTEM and open Administration.
2. Switch to the Configuration tab and expand xsengine.ini | public_urls.  Double-click on "public_urls" to invoke the Change Configuration Values dialog.
   
   
   
   
3. Click the Databases drop-down box, select the tenant database(s) you want to configure and click OK.
   
   
   
   
4. Provide the new URL using the alias hostname and click Save.
   
   
   
   

You are now able to XS engine for the tenant database (e.g. http://mdc-tn1.internal.sap.com:8000) without getting the 503 error.

If you want to use SQL instead of HANA Studio, you can execute the following statement in a SQL Console.  Don't forget to connect to the SYSTEMDB (not the tenant database!) as the SYSTEM user.

ALTER SYSTEM ALTER CONFIGURATION ('xsengine.ini', 'database', 'TN1')
SET ('public_urls', 'http_url') = 'http://mdc-tn1.internal.sap.com:8000'
WITH RECONFIGURE;

Re-Configure HTTP Access for the SystemDB

If you now try to access the SystemDB XS engine using the same URL as before (e.g. http://mdc.internal.sap.com:8000), you'll find that you'll get (oh no!):

503 Service not available

The documentation clearly states that the URL for the SystemDB changes once you've configured the URLs for the tenant databases.  Instead of using the FQDN, the SystemDB XS engine is available under http//<localhost>:80<instance>.  In our example, this means that the XS engine would now be found at http//mdc:8000.

What you need to do now is execute the following SQL statement to re-configure the SystemDB URL to use the alias host name:

ALTER SYSTEM ALTER CONFIGURATION ('nameserver.ini', 'system')
SET('public_urls', 'http_url') = 'http://mdc.internal.sap.com:8000'
WITH RECONFIGURE;

Note that if you try to make this change in HANA Studio by editing the xsengine.ini configuration file, the change will not take effect.

Voila!  You now have access to the XS engines for the SystemDB and tenant databases using the alias hostnames.

Final Thoughts

You can spend a lot of time trying to figure out how to enable HTTP access to your SystemDB and tenant databases.  Usually once you configure the tenant databases properly, you scratch your head determining why the SystemDB URL no longer works.  The procedure to fix this is quite straightforward, once you know where to look.  At a minimum, following these steps will allow you to open each tenant database's HANA Cockpit, as that's also a HANA XS application.

First of all, thank you for reading part 2 of my blog on CDS. For a basic example, please read part 1 of this blog.

On part 2, I wanted to expand into creating another table with an association. On this post, I will explain how to create a simple 1 to 1 Association, however, CDS can further support associations such as 1.. many, etc.

1) On my new table HCCAsugEvent below, I created a column EventPresenter with an association back to HCCAsugPresentations (my first entity). Notice that the key on my HCCAsugPresentations will become my key for the associations. I did not have to explicitly state it but it was is understood on my next step.

2) I went back to the workbench Catalog and opened the SQL Console. I ran a Select query to verify the table was indeed created.

3) After the table was created, I inserted a few records (no picture of the insert, but it was easily done by right clicking on the table and then selecting "Generate insert". One nice thing about the workbench I discovered during the exercise is that I can double click on the table and its definition will be shown. I knew about this feature from HANA Studio but it was also nice to see it on the workbench (Thanks SAP teams!)

My next step was to create a view for the 2 tables I created. Initially, I created a select query with an inner join. I knew this is a way to do it in SQLScript, but I wanted to also achieve this from my CDS view involving at least 2 tables.

Next is my approach to create a similar view via CDS. The great thing about creating an association via CDS and trying to do a JOIN on my view is that my entity knows about the association so that I do not need to invoke my associated table. The column that has the relationship knew about the association and it lets me callt he columns from the other table via its COLUMN_ON_TBL.COLUMN_FROM_ASSOCIATED_TBL property as shown below. I added at least one more column from my joined table just to showcase the easiness of creating views via CDS. I am becoming a fan!

Here is the output

Thank you for your time reading this blog and feel free to post your questions or share your thoughts about your CDS experiences.

Here are some helpful links I used while writing this blog:

create a CDS document

create a CDS view

as per request... someone asked me how to expose this view or data model via xsodata , here is how on my next blog post. Good luck on your scenario!

Hello, In this blog post, I will showcase how to create a table and a view via the Core Data Services (CDS).

In my example I will be using the HCP trial landscape and the Web Development Workbench editor.

1) Create a CDS file ( .hdbdd extension)

2) Add some (correct) content and activate the file

3) Go to your schema and open the tables folder to see the newly generated table.

4) Perform a select statement - as expected, initially your table will be empty

5) Right click on the table, select “Generate Insert”   or type it (if you want to make spelling mistakes)

6) Your generated insert statement is opened on a new tab. then, you can save it as a .sql file (selct a location on your package)

7) I opened the sql file, copied and pasted into the SQL console so i could run it. Notice console output successful insert message, "Rows affected 1" after I ran my insert statement.

8) after I was able to insert my statement, then I went back to my select statement and I ran it again to verify my record was in fact inserted. As a result, you can see my first record in the table

9) After a few inserted records, we can see the table again with more rows populated.

10) Since I have a table, then I want to create a simple view. I went back to my CDS file and I declared a view. Then I re-activated my file.

11) Once the hdbdd file was activated, then I went back to my Catalog and opened the Views folder where my newly created view was saved.

12) Once again, I did a right click on the view and generated a select statement.( I avoid typing my SQL as much as possible so that I do not end up having to type it over and over before I get the correct spelling but up to you to try to get the name right the first time.

13) My select statement was generated

14) After the statement was generated I ran it (I changed it a little but the one above should have returned the same number of records since I only have 6 on my example. See output below

* Be careful if you are doing this on a table with a lot of records as you may select *

Thins to remember about CDS:

1) Stored procedures and sequences cannot be created in CDS (as of SP10)

2) CDS files may contain one or more entities or views and when it is activated, your entities, views, etc are directly created as design time objects on your schema and are ready for use.

3) HCP trial ( in your Web Development Workbench) is very helpful for learning how to use CDS ( as you may also use HANA Studio)

Things to forget:

1) Forget to re-run SQL statements when promoting your code from one environment (such as dev) into another environment (such as QA or Prod). CDS plays very well with HALM during migrations.

2) Forget to think you are alone on this journey, you have an entire SCN group behind you

3) (I forgot 3 - i am working on the next exercise on my Workbench, aren't you?)

Again, this is a very simple document explaining how to create a table and a view via CDS. Please share your experience or struggles with CDS. Happy programming. For a more advanced scenario, please read part 2 of this blog where I showcase a view with a 1 to 1 association on a table

Here are some helpful links I used while writing this blog:

create a CDS document

create a CDS view

I was not planning on this post but I got a few questions from some colleagues as to how to expose CDS views via XSOData so here it is

As always, thank you for reading this post.

Before creating the xsodata service, let's start by assuming you have created a CDS document with a table / view and you are ready to expose your data model. If you have not done those steps, please refer to my first CDS blog to create a CDS document and build a table / view

1) The first step is to create an .xsaccess file so we can expose our service. Make sure you set the property expose: true and your also need to set its authentication method. In my case, I do not want to add authentication as I am using HCPtrial, however, you may use null (for no authentication), Basic (for username and password), Form, etc. Please use the XS Admin tool for other authentication methods such as SAML, X509 certs, etc. check out the official documentation for xsaccess files

Also create an empty file .xsapp

2) create your .xsodata file (I am exposing the view I created on my blog 2) when exposing an end point on your xodata service, make sure your endpoint contains a KEY, otherwise, you may use a dynamically generated key as I did below

3) once you created your .xsodata file and have successfully activated it, run your service by clicking on the green icon with an arrow (see screen shot above). Your service will run on a separate tab on your browser and will display the XML of your service. One thing to note here is that my endpoint on the .xsodata service is exposed as "EVENTS" and that's exactly the endpoint on my XSOData service on the browser (screenshot below)

4) your next step would be to explore the end point, and the data that will be returned, etc. I am not able to run my service due to insufficient privileges since i am running this service on my HCP trial account, but if you are able to activate your service and you can query it from your browser, then you should be able to see the records being returned by your odata service.

5) for a JSON formatted response, please include $format=json at the end of your service URL.

a). URL_HOST/path/to/service/ENDPOINT?$format=json

you can also perform other query operators by appending query string paramaters to the end of your URL endpoint.

here are a few:

$top=1  for the first record

$filter=COLUMN eq 'value for filtering results where COLUMN equals 'value'

$format=json for your JSON formatted response

$orderby=COL_A for ordering the response by COL_A

for more OData operations and syntax, please visit the odata official documentation

thanks again and hope this has been helpful for others. As always, please share your experiences and struggles and feel free to ask questions.

The UI theme designer is SAP's browser-based tool for cross-theming scenarios.Create one consistent theme that applies to various SAP UI clients and technologies.Theme designer supports only three deployment options (ABAP, Enterprise portal and HCP) but you can apply custom theme which you have developed using UI theme designer accross various UI clients and technologies.Below picture gives you an overview on supported platforms and UI technoloies.I would like to bring on more clarity on the steps being followed to apply a custom theme for any HANA XS based applications on both cloud and on premise which uses SAP UI5 as front-end technology and hence this blog.. You must also take care for the UI5 versions. They should be the same on both systems. Otherwise the theme can be broken.

Application used in this blog: SHINE Application

Prerequisites:

Create your custom theme as per your requirment. Export your theme(.zip file), the exported theme is saved on your local machine. Please follow few blog below to create custom theme,

• http://scn.sap.com/docs/DOC-52952
• http://scn.sap.com/community/developer-center/front-end/blog/2013/09/19/create-and-apply-custom-theme-to-sapui5-application
• http://scn.sap.com/community/developer-center/cloud-platform/blog/2015/08/14/setting-themes-for-sapui5html5-applications-via-url-parameter-and-bootstrap-configuration-in-hcp-trial

Step1:- Create and Import Custom Themes to HANA System:-

1. You can create custom themes using the UI theme designer and import these themes into your SAP HANA system. In the UI theme designer tool, create and export a custom SAPUI5 theme that you want to use for your XS based application. A .zip file containing the exported theme is saved on your local machine.
2. Import the exported theme into your project in SAP HANA Studio/SAP HANA WEB IDE(Copy the contents of the .zip file into your project)
3. To register the theme in the THEME database table using the table import mechanism, create the following files in your project:

myTheme.hdbti

myTheme.csv

   4. Activate the .hdbti and .csv files by choosing Start of the navigation path and activate it.

Step2:- Applying Theme Via  Bootstrap Configuration

Insert the following in the SAPUI5 bootstrap script tag:(index.html file)

data-sap-ui-theme:  Enter the theme ID you have provided when you exported your theme from the UI theme designer to your local hard disk.

data-sap-ui-theme-roots:  Enter the theme ID and the application path you used for your mapping in the following syntax:

{"<themeId>" : "/<application path>/UI5/"}

Now check your HANA XS based application with custom theme applied

Further Reading please refer below documentation,

http://help.sap.com/saphelp_hanaplatform/helpdata/en/96/ae16b69a054cfaafed1be567885e71/content.htm

https://sapui5.netweaver.ondemand.com/docs/guide/5a2df7b50f714d8bb5e9b519a88aa1d6.html

http://scn.sap.com/community/developer-center/cloud-platform/blog/2015/08/14/setting-themes-for-sapui5html5-applications-via-url-parameter-and-bootstrap-configuration-in-hcp-trial

https://help.hana.ondemand.com/theme_designer/frameset.htm

Get back to me if you need more information

Part 1: Setup HCP account and IDE, twitter account and download first tweets

As a student in SAP HANA Product Management I had to explore what can be done on HCP trial landscape, what runs out-of-the-box, which SAP HANA features are available – and which business needs can be met, in the end? So, here are the results.

In this series of blogs I am going to explain how to build an real-time social media analytic application using the SAP HANA cloud platform (HCP). I wanted to make use of predictive algorithms. A hotspot for predictive is marketing. Therefore, I chose this topic. Of course, it is not ready-to-use, like a product. But it is “ready-to-copy-and-learn” and I named it SmartApp. While digging deeper and deeper, of course I learnt a lot through a multitude of other blogs and SAP HANA Academy videos. Thanks to all of you! (I do mention them below.)

It is an application that you can follow along yourself. However first let’s have a quick look at the overall scenario of the SmartApp. We are going to access some real-time social media information. I would like to take twitter as the example so we can get information on specific tweets that we want to look for and will pull those tweets out in real-time using the twitter API. The idea for this use was fueled by a nice blog from a Chinese colleague of mine, Wenjun Zhou:http://scn.sap.com/community/developer-center/hana/blog/2014/11/23/real-time-sentiment-rating-of-movies-on-sap-hana-part-6--wrap-up-and-looking-ahead- kudos to you for this work!

We will process those tweets and load them in real-time into the HANA database. However the database is hosted on HCP. It was my ambition to use an HCP developer trial edition for as long as possible (until I need a feature that is only available on a dedicated HANA instance). Once we loaded the data into the database then we can do some further analysis on how to group or segment the different types of tweeters, maybe we to group similar people together, for example the group of people expressing positive or negative sentiments towards the product or company, and it would also make sense to differentiate whether they are influential because a lot of people follow them and their tweets get retweeted. We could identify that group so we can target some education program or reach-out program to those people. So, the idea was to do some predictive analysis in HANA itself in order to segment, cluster or to group similar kinds of people sending tweets. And finally, I wanted to make the results available through web services i.e., OData, XSJS native web services within the HANA platform in order to build a HTML5 application that could run on desktop and mobile devices to look at the analysis results. That’s the overall scenario of the SmartApp.

I don't want this blog to stretch like a chewing gum, so I divided the whole story into 3 parts:

1. Setup HCP account and IDE, twitter account and download first tweets

2.Refine processing of tweets with text Analysis, building web services and setting HANA Cloud Connector – this is the “heavy lifting” on SAP HANA from my point of view!

3. Do the UX work – use Analytics tools – make it nice

In the following sections, of course there is a lot of overlap with many existing blogs out there. Thank you to everybody I learnt from. Here are just a few of them:

• http://scn.sap.com/community/developer-center/cloud-platform/blog/2013/10/17/8-easy-steps-to-develop-an-xs-application-on-the-sap-hana-cloud-platform
• Live3 - SAP HANA Cloud Platform Tutorials for Advanced Real-Time Social Media Analytics
• http://scn.sap.com/community/developer-center/front-end/blog/2015/03/18/how-to-consume-a-hana-xs-datasource-on-the-hana-trial-landscape
• http://scn.sap.com/community/developer-center/front-end/blog/2015/01/12/accessing-hana-xs-odata-in-web-ide-and-beyond
• http://scn.sap.com/community/developer-center/cloud-platform/blog/2014/06/19/debug-your-sap-hana-xs-application-on-trial

Prerequisites

Before we start to build the application, we need to register for HCP trial developer account and create Hana instance within the trial account (recommended to give instance name as “dev” and Database system as HANA XS (shared)since we can refer it in many applications).

• We need Java Virtual Machine (JVM) to run the eclipse hence install JDK (version 7).
• Install the Hana Cloud Platform SDK by clicking on the java web version in https://tools.hana.ondemand.com/#cloud
• Install Eclipse (Luna or Mars).
• Install SAP Development tools for Eclipse: Select SAP HANA Cloud Platform Tools and SAP HANA Tools to install the whole toolkit.

Open SAP HANA Administration console perspective by clicking top right cornerof eclipse.

Find API keys

• A developer account for Twitter Developers. If you have a Twitter account, just login. If you don't have, sign up here
• Create a twitter application to obtain the API keys.
• Click herefor Connecting to SAP HANA Schemas via the Eclipse IDE.

SET SCHEMA

First, identify the schema name to put application specific artifacts like tables, views:

To Identify and set Application schema keep DEV_xxx schema selected and hit onSQL button.

Make sure to set the database connection by clicking on.

Copy the following statements to the sql console in eclipse:

To Identify application schema,copy:

SELECT SCHEMA_NAME FROM "HCP"."HCP_DEV_METADATA";

To Identify current user,copy:

SELECT CURRENT_USER FROM DUMMY;

To Verify predictive analysis library is available,copy:

SELECT * FROM SYS.AFL_FUNCTIONS WHERE PACKAGE_NAME='PAL';

To Set application schema - paste in value below,copy:

SET SCHEMA NEO_;

Select & Run the statement to identify application schema:

SELECT SCHEMA_NAME FROM "HCP"."HCP_DEV_METADATA";  execute by clicking .  As a result, the schema name is shown.

Then right click on the schema name and copy the cells. To set application schema – Paste the value below as

SET SCHEMA NEO_xxxxxxxxxxx; execute it to activate the schema.

Create Tweets table executing the below code:

CREATE COLUMN TABLE "Tweets" (

             "id" VARCHAR(256) NOT NULL,

              "created" TIMESTAMP,

               "text" NVARCHAR(256),

               "lang" VARCHAR(256),

               "user" VARCHAR(256),

               "replyUser" VARCHAR(256),

               "retweetedUser" VARCHAR(256),

               "lat" DOUBLE,

               "lon" DOUBLE,

                PRIMARY KEY ("id")

);

Now we can find the tweets table in the schema (NEO_xxxxxxx) that we set initially.

Now install node.js to access both twitter API and HANA Database very easily that connects together.

To download the code of SmartApp from Git hub please click here.

Open downloaded SmartApp -> nodejs-> app.js in the editor may be notepad++.

In the code snippet we can see a variable called twitter where we actually specify the consumer_key, consumer_secret, access_token_key, access_token_secret. If you remember we already got this information when we registered our application in twitter. Now copy and paste the consumer_key into our file. Similarly do the same for other values also as shown below.

var twitter = new twitter({

       consumer_key: 'xxxxxxxx',

      consumer_secret: 'xxxxxxxxxxx',

      access_token_key: 'xxxxxxxxxxx',

       access_token_secret: 'xxxxxxxxxxx'

We have a second file called package.json in the nodejs with the list of dependencies like express, request, oauth, twitter, hdb that are used in our application. Now we can install all these with an npm and node installation command. Now click shift+rightmouse button on the nodejs folder in the SmartApp to open a command prompt in that nodejs folder then use “npm install” command to install all the dependencies that we needed in our application.

Then Restart the system (computer) so that nodejs could run without errors.

Go to nodejs folder should be able to see node_modules folder which actually consists the code for the modules setup and installed.

Setup the Tunnel

Let us setup the Database tunnel to connect directly to the HANA database using p-number trial id (pxxxxtrial). In order to setup the database tunnel open a command prompt on the tools folder in neo-java-web-sdk by shift+right click mouse button. Now execute the command “neo open-db-tunnel –h hanatrial.ondemand.com –i dev –a pxxxtrial –u pxxxx” (please enter the command manually don’t copy!) by replacing the account name with your pxxxxxtrial account and username with the pxxxx number.

After executing the command enter the password that you gave during the trial account creation. The tunnel opened and gives us the properties to connect to the schema.

Now go to the app.js file, edit the port as 30115 then copy the user, password, Schema name properties and paste in the app.js file respectively as shown below.

var hdb = hdb.createClient({

            host     : 'localhost',

            port     : 30115,

            user     : 'DEV_xxxxxxxxxxxxxxxxx',

            password : 'xxxxxxxxxxxxxx'

});

var hdb_schema = "NEO_xxxxxxxxxxxxxx";

Now let’s start the application by simply going into command prompt running in the nodejs folder. Goto the downloaded smartApp -> nodejs folder then shift+right click then select open command window.Type “node app.js” as command, application will run and we can see two messages as shown below.

Goto the Chrome browser enter “localhost:8888/do/start?track = productname or companyname”. 

Note: Enter the product or company name that you would like to track or make analysis. 

Now we can see in nodejs command prompt that tweets are loading into HANA database. Go to the tweets table in eclipse and refresh by clicking on top right corner. We can see some tweets that are loaded in to table.

If you want to stop loading the tweets go back to chrome browser then type “localhost:8888/do/stop”.

Note: While working with tunnel it must remain open for nodejs in order to be connect to the Hana instance. Since it’s a trial account this tunnel only lasts for 24 hours at that time you need to reopen it. However it’s fine for development and testing purposes. When using Productive HCP able to let things run 24/7.

Thanks for reading. Hope you enjoyed the blog….

Next Steps

In the next blog will perform some text analysis on Tweets data, Setup SAP HANA Cloud Connector and Configurations, Building web services using HANA Native development and setup authorizations. I will insert the link in here as soon as it is available!

When you’re preparing to install dynamic tiering, make sure to review the hardware and software requirements so you can plan your system accordingly.

Software Requirements

Dynamic tiering SP 11 supports the following operating systems:

Operating Systems on Intel-based hardware platforms:

• SUSE Linux Enterprise Server (SLES) 11 SP2, SP3
• SUSE Linux Enterprise Server (SLES) 12
• Red Hat Enterprise Linux (RHEL) 6.5 or higher

Operating Systems on IBM Power system platform:

• SUSE Linux Enterprise Server (SLES) 11 SP4 for IBM Power

However, dynamic tiering needs to be installed on the same operating system as HANA. For example, you can't have HANA running on SLES and dynamic tiering on RHEL, even though both are supported.

Hardware Requirements

Dynamic Tiering doesn’t have any specific hardware requirements. However, if you want to add a standby host, it needs to have the same hardware layout as the worker host that it's providing failover support for.

To see the most up-to-date list of system requirements for dynamic tiering, check the SAP HANA Dynamic Tiering Hardware and Software Requirements topic in the SAP HANA Dynamic Tiering: Installation and Update Guide.

Related Documentation on the SAP Help Portal:

• SAP HANA Hardware and Software Requirements

SAP HANA Customers & Partners, join us for the 2016SAP HANA Operations & Developers Expert Summit! on May-24^th, 25^th at SAP America, Newtown Square, PA. Based upon the success of 2014, 2015 Operations & Developers Expert Summit, SAP HANA Product Management team is inviting IT Architects, Administrators, DBAs, and Developers this year again to the 2016 SAP HANA Operations & Developers Expert Summit. We wanted to hear from you firsthand about your SAP HANA Platform implementation experiences, use cases, and challenges in operations and development topic areas, what is working and what is not working, etc.

So if you have worked with any of the Operations and Development topics in SAP HANA platform areas, please join us on May-24^th or 25th in Newtown Square, Pennsylvania. The plan for the event is to have a quick update presentation for each topic area of the SAP HANA Platform, followed by a round of feedback sessions with the product manager responsible for a specific HANA topic and let them know what are your greatest successes, and challenges with SAP HANA Platform Operations and Development areas. This is an exclusive event to get access to the SAP HANA Product Managers who can help you to influence the product strategy and roadmap based upon your direct feedback to them. Don’t miss this opportunity to meet the experts in person at the event.

During the event, you will be interacting with a small number of experienced hands-on experts, networking together with various HANA customers and providing direct feedback to SAP HANA product managers. To this end, the number of registrations will be limited, where all the registration request will be given careful consideration and we will contact you with a confirmation email to attend the event and the next steps.

After spending some time trying to load data using a C# ETL loader we created, I noticed that SAP exposes a "Bulk load" option in their ado.net driver. I couldn't find any examples on using it, so I figured I would post an example. Performance is pretty good, with the BatchSize = 1000, I am able to load 1 million records in about 10 seconds.

        private string dbConnectString;        private HanaCommand dbConnection;        public void LoadData(DataTable dt, string tablename, string[] columns)        {            string[] fullName = tablename.Split('.');            dbConnectString = ConfigurationManager.AppSettings["DatabaseString"];            HanaConnection dbConnection = new HanaConnection(dbConnectString);                      dbConnection.Open();            HanaCommand da = new HanaCommand("SET SCHEMA " + fullName[0], dbConnection);            da.ExecuteNonQuery();                                        HanaBulkCopy blkcmd = new HanaBulkCopy(dbConnection);            blkcmd.BulkCopyTimeout = 1000;            blkcmd.DestinationTableName = fullName[1];            HanaBulkCopyColumnMappingCollection mappings = blkcmd.ColumnMappings;            for (int i = 0; i < columns.Length; i++)            {                mappings.Add(new HanaBulkCopyColumnMapping(i, columns[i]));            }            blkcmd.BatchSize = 1000;            blkcmd.WriteToServer(dt);            blkcmd.Close();            dbConnection.Close();            blkcmd.Dispose();            dbConnection.Dispose();        }

A few notes about implementation and some things I found:

1.The datatable has to have matching datatypes for the columns, but the rows can be all strings, so when you initially build your datatable you have to do something like this for each column:

DataTable dt = new DataTable();                                        
DataColumn dc = new DataColumn("col" + j.ToString(), typeof(double));
dt.Columns.Add(dc);

And specify if it is a double, string or DateTime etc.

2.The BatchSize has some strange behavior. I didn't play with it much, but if you don't define it then performance is really terrible. Not any better than inserting one record at a time. I found 1000 gives great performance, but I bet it could be even better, so if someone is able to do some metrics that would be great.

3. I am using the HanaBulkCopyColumnMapping in my example, but it is completely optional to use. If you don't provide the mapping, then column 0 in the datatable gets mapped to column 0 in Hana 1:1, 2:2 etc. The mapping is nice of course, because then you don't have to have the same number of columns or the correct order, I just pass the tables columns names that I am loading in as an array string in the example.

4. You will notice that I switch the Schema before loading. If you don't do that, then even if you pass in the fully qualified name (ex USERS.USER) the bulk loader will still try to load into its own schema.

5. The connection string is a pretty simple string that consists of something like this:

"Server=hana-server:30015;UserID=username;Password=passcode"

Following the excellent openSAP course Software Development on SAP HANA (Delta SPS 11), I wanted my own SAP HANA XS Advanced environment.

However, since at the time of this blog, there has been no SAP HANA Datacentre Service Point released for HANA SPS 11 yet, I had to find a sandbox environment. Please check SAP Note 2021789 - SAP HANA Revision and Maintenance Strategy, whether this is still the case when you read this.

Luckily, I found a no longer used HANA SPS09 PoC System, that I could use, which of course still had the traditional service structure:

Upgrading the HANA Database including the XS Advanced Software Components was fortunately straight forward. You start the HANA Lifecycle Management from the SPS11 installation media:

Interestingly, SAP HANA Smart Data Access had been pre-selected and could not be removed, but I just added the components, that I wanted, and later it turned out, that I could deselect it:

Subsequently you will have to provide some passwords. Especially important is the XSA_ADMIN, since this is the user we will log-on later, as well as the Customer Organization Name and Customer Space Name:

From there, the installation should run smoothly:

With eventually all SAP HANA components updated:

As a result, we see the new service structure:

And subsequently, we can log into SAP HANA XS Advanced and for example list the deployed apps, using the XSA_ADMIN user and password provided before:

Part 1 of this series focused on how the new SAP HANA Data Lifecycle Manager features helped to utilize automated processes to tier the data from HANA to Hadoop.  Now that the data is in Hadoop we will look at how to start analyzing this data with SAP HANA Vora.  Before we get too far let me give you a quick high-level overview of SAP HANA Vora….

“For digital businesses that want to infuse business decisions with valuable context from new data sources, SAP HANA Vora is an in-memory query engine that plugs into the Apache Spark execution framework to provide enriched inter- active analytics on data stored in Hadoop. It lets you combine Big Data with corporate data in a way that is both simple and fast. “

Vora is a key component in the SAP Big Data Strategy for the digital enterprise.  As part of the data platform Vora helps to bridge the digital divide for Analysts, Developers, DBAs, and Data Scientists.   This blog will focus on the Data Scientists.

Vora will Transform Big Data analysis with an in-memory computing engine for Hadoop.

Correlate Hadoop and SAP HANA data for instant insight that drives contextually-aware decisions.

• Organize massive volumes of unstructured data into data hierarchies
• Make OLAP modeling on Hadoop data accessible to everyday users with popular SQL language
• Simplify Big Data management with data federation for SAP HANA and Hadoop landscapes

Now that you have an idea of what SAP HANA Vora is, let’s dive right in.   We are going to look at how a data scientist could utilize the data that was tiered from HANA-->Hadoop.

In Part 1 we specified the DLM destination as “Spark destination”. In order for us to be able to work with the data we first need to understand a few things on how and where the information is stored in Hadoop.  The destination parameter is specified for the spark controller in the hanaes-site.xml file.  In the below screen shot you can see for this system the destination in HDFS is /sap/hana/hanaes/warehouse.

In part 1 we actually ran the DLM job.  As a result if we look at the target HDFS destination we can see the following files that have been created by DLM.

The thing to note here is DLM creates Parquet files when relocating data to Hadoop.  What is Parquet? In short …

“Apache Parquet is a columnar storage format available to any project in the Hadoop ecosystem, regardless of the choice of data processing framework, data model or programming language.”(More information on parquet here).  This is a perfect match for Hana as Parquet is a columnar storage format. 

There are a few options to work with Vora.  Zeppelin, the Spark shell, or Beeline.  At the risk of this blog getting to long, I’ll try to show an example of all 3.

Zeppelin

What is Zeppelin? 

“A web-based notebook that enables interactive data analytics.

You can make beautiful data-driven, interactive and collaborative documents with SQL, Scala and more.” (More information here)

Configuring Zeppelin to use the Vora interpreter takes some setup (I’ll save this for another blog if anyone is interested).  For the purpose of this blog I’ll assume you have zeppelin configured and working with the Vora interpreter.

1 – Open Zeppelin in your browser

2 – Create a new note and type in your create table statement.  The Vora interpreter is utilized by specifying %vora.

3 – Now that the table has been created, SQL statements can be executed against it.  If you remember part 1 of the blog we relocated 100,740 records from Hana to Hadoop.  We can do a count (*) on the vora table to validate all the relocated records are accounted for.

4 – At this point you are free to run any query against the Vora table.  I have done a simple sum and group by as seen below.

Spark Shell

If you don’t need to visualize what you are working on in a flashy way then you can simply use the spark shell to run and review your results.

1 – Start the spark shell. You can find it in the Vora bin directory.

2 – Import org.apache.spark.sql

3 – Create the Vora table. The syntax is the same as what we used for Zeppelin

4 - Similar to zeppelin we can run the sum select using the standard SQL syntax.

1. sqlContext.sql( s"""

SELECT

    SQUARE_ID

    ,SUM(cast (SMS_IN as double))as SMS_IN

    ,SUM(cast (SMS_OUT as double))as SMS_OUT

    ,SUM(cast (CALL_IN as double))as CALL_IN

    ,SUM(cast (CALL_OUT as double))as CALL_OUT

    ,SUM(cast (INTERNET_TRAFFIC as double)) as INTERNET_TRAFFIC

FROM

    tmp_cdr3

GROUP BY

    SQUARE_ID

order by CALL_IN desc

""".stripMargin

).show

Result:

Beeline

We use Beeline, a JDBC client, to register tables created in SAP HANA Vora in the Thrift server .  This method will allow SAP Lumira to connect to the Vora tables.

1 – Start beeline.  This is located in the /vora/bin directory.

2 – Submit the create table statement; again this syntax is exactly the same as we used in the previous 2 examples.

3 – In order to connect with Lumira, the thrift server needs to be started.  This will enable a connection from Lumira to the Vora table. The thrift server is located in the vora bin directory.

4 – Next we move to Lumira. Create a new dataset.  The dataset connection to Vora is done using the Generic JDBC Drivers as seen below.

5 – Once the JDBC data source is selected we need to set the JDBC URL and the JDBC Class.  The class that we utilize will be com.simba.spark.jdbc4.Driver

6 – Once we have successfully connected to the Thrift Server via the JDBC connection a list of the Vora tables should be available to query.  See below the cdr_detail Vora table that we created in beeline.

7 –Standard Lumira visualizations and analytics can now be created against the Vora table.

Conclusion: This is our first look at analyzing the Hadoop data with Vora.  You should now have a sense as to how Vora is bridging the digital divide for Data Scientists.

Next Steps: In part 3 of this blog series we will update our code to show how Vora is utilized to combine both Hot data in HANA and the cold data in Hadoop.  The next part will really showcase how Vora can be utilized to democratize data access in the enterprise.  Following that we will take a look at how we democratize and bridge the gap from the perspective of a data analyst.

The Data Provisioning Agent hosts all SDI Adapters and acts as the communication interface between Hana and the Adapter. As such the DP Agent version has to match the Hana version. Which version to use is best found out by looking at the PAM (link) for Hana SDI, the Essential Information document.

The DP Agent can be installed anywhere and one Hana instance can have many agents configured. Hence there is a great freedom. Some adapters might have certain requirements however, e.g. a File Adapter can read all files it has access to. In order to read a file from a local disk drive, the File Adapter and therefore one Agent has to be installed on this local computer.

see architecture

The SDI Agent can be downloaded as SAR file from SAP Service Marketplace like any other SAP application, provided it was licensed.

As the Adapters are written in Java, it is a good idea to check if Java is installed and what version.

The actual installation is straight forward. The hdbsetup.exe guides through the installation steps.

Since this is a network service, in case of Windows the domain\username (or hostname\username for local users) has to be provided. And then the service does listen on the two TCP/IP ports, by default 5050 and 5051.

When the installation is finished, the Windows Services should have a "SAP Hana Data Provisioning Agent" running.

Now this new agent has to be made known in Hana by executing the create-agent SQL command.The same can be accomplished by running the dpagentconfigtool.

In order to execute SQL commands the tool has to connect to Hana using the JDBC driver (or https in case Hana is in a different network with a firewall inbetween, e.g. a cloud instance) and the user has to have the required permissions.

Once connected the tool executes the CREATE AGENT <agent name> PROTOCOL 'TCP' host <hostname or IP> PORT 5050; command to register an agent. Hence an arbitrary agent name has to be entered and the local hostname.

Note: When using the TCPIP protocol, it is the Agent listening and Hana initiating the communication. Therefore the hostname has to be know to the Hanas system. In larger organisations that means the full qualified hostname has to be used often. Or the static IP address of the Agent computer, if there is one. This gets especially tricky in case the Agent runs on a computer with DHCP.

At the end the dpagentconfigtool should show that it is connected to Hana successfully, the agent name is registered and the agent is running and listening as shown below.

Hello everyone, thank you for taking the time to read this short blog. I wanted to share a couple ways to load data into HANA tables as I prepare for a demo where I am starting from an empty slate.

For data loads involving more than one-time loads, there are other tools/techniques used which are not on the scope of this blog such as (Data services- ETL tool, SLT (trigger based replication), XS Jobs running on schedule, etc)

My demo will involve creating tables, views, and other information models as I shared with you on my CDS blogs a few weeks ago.

Now, I need to load some data into my tables (yes these are dummy tables and not what I will use for my true demo but you will get the idea)

My first approach to load data is via the CDS table import - official documentation - This approach is very clean and straight forward to do.

1) I created my target table "testTable" with 3 columns, namely, ID, col1, col2

2) Created a sample csv file and uploaded it to my project folder (just dragged and dropped and activated it there) - below you see the file and the sample data  (6 records for demo purposes)

3) once the target table was created, and the sample data is uploaded to the project, we need to create an .hdbti file (table import) which will do the trick for us.

* if you are using HANA Studio, you may click on New > Other > (table import wizard) I typed "impo" and just followed the wizard steps (provided a name, selected a folder in my project, etc)

The file below gets created and then you need to provide the correct information for the target CDS table ( which we created on step 1) , the file that contains the data (step 2) and the delimiter field (default is , but it could be anything else ) once this is completed, activate your .hdbti file

After the file was activated, I can see the table was in fact populated

My next attempt to perform data loads into the same table (append to existing content) is to use the Data Modeler perspective and use the (Window > Show View) Import Wizard so that I do not have to load and activate the flat file into my package and only doing a one-time import following the wizard.

1) Open the SAP HANA Modeler perspective  / Quick view tab

2) Open the SAP HANA Content  folder >  "Select Data from local file"

3) Select your target system ( sometimes you may have more than 1 system depending on hoe many connections you have opened)

4) Select your file, target table, delimiter, specify whether or not your first row contains the header (column names), etc

5) perform a column mapping (drag from the left panel columns to the right panel columns).

notice also that the data preview on the bottom panel is available (data from flat file)

Click Finish

Return to your SQL console and run the select statement to insure the data was loaded

if you noticed on step 1 that my ID column was a key. My imported records cannot contain a duplicate record with the same ID. I did this on my first attempt and received an error due to the constraint was violated, therefore, I went back to my flat file, and made sure the IDs were different and finally I was able to load the data without issues after that change.

Just to conclude, this short demo was about doing some initial data loads for demo purposes. For data loads involving more than one-time data loads, there are other tools/techniques which I did not discussed on the scope of this blog. such techniques/tools are Data Services - ETL tool, SLT (trigger based replication), XS jobs running on a schedule, etc.

Thank you again for reading this blog and please feel free to ask questions or share your experiences.

DevOps, and particularly Continuous integration (CI) and continuous delivery (CD), are terms which has been adopted widely and over the past few years in the software development industry. It's use is currently creeping outside the scope of development and into various departments wanting to decrease long delivery cycles and increase product iteration. For those unfamiliar with the terms in the software space, or premise behind the idea: CI/CD is essentially the automation of the delivery of your software development projects into production and the broader goal is to bring Development and Operations closer together. In this blog we will mainly focus on CD as it pertains to HANA XS development.

Inside SAP IT, and like a lot of other IT departments, we are trying to increase and simplify the deployment of our HANA XS projects and move to a more agile approach in delivering solutions for our internal customers.

Firstly, some background: As many of the XS Developers out there know, the HANA repo is not necessarily the easiest "file system" to work with. The fact that data is stored in the DB, in a propriety format and each file needs to be activated, makes it tough to automate basic operations like moving or copying. In order to work around this topic, we decided that all of our code deployments were going to be done using HANA’s preferred native “packaged” file type known as delivery units (DU). These contain the entire active code base of a project (or subset of a project), changed as well as unchanged files.

In the past we manually deployed code to each of our instances individually, this required manual intervention and developer/operations availability which we hoped we could streamline. The DU import process we decided to use is a feature which was introduced in SPS09 through a command line tool called HDBALM. This allows any PC which has the HANA client installed to import packages and delivery units to a HANA server. While there are options to commit and activate individual files from a traditional file system folder system (using the FILE API), we felt the benefits of a DU were better suited to our use case (for example, hdb* objects which may need to be created in specific order).

Since we have the ability to deploy our code to our various servers using HDBALM, we needed something to get the job done! We used our internal instance of Atlassian Bamboo. We use this server for our HCP HTML5 and Java apps which make it a logical choice to keep our projects grouped to together. Our build agents are redhat distros and have the HANA client installed. We also copy over the SSL cert since our hana servers are using https and these are needed for hdbalm.

In this case, and example, our landscape is relatively simple with a Dev, Test and Production dedicated HANA HCP instances.

We store our “ready for deployment” delivery unit on our Internal Github instance, we do this so that the file is version controlled and also visible and accessible to our dev team. The premise is that the dev team would push a version of a delivery unit after their sprint, and its ready for deployment to Test. This could easily be a file system as well. However, we like to use the push/tag/release of our Github repo to trigger the build deployment.

FYI: Bamboo is a great tool and a nearly zero cost barrier ($10). If you are considering a build server which has a quick and easy installation (*nix) and setup, I would highly recommend it.

Since we have gone through some of the logical details, here are some technical details covering the topic:

As mentioned previously, our build process is trigged by a new delivery unit being committed to our github repo. Our bamboo build agent picks the build up, saves it on the build server and deploys it to our Test instance. A email is sent to our dev and test team with the results. Before the import process, we check the version identifier on the existing delivery unit which was on the server, and we subsequently check it again after the import for comparisons and decide if the import was successful (along with the results of the import statement)

(Keep in mind the below commands include $bamboo_variables) but would work just fine to replace them with actual values.

export HDBALM_PASSWD="$bamboo_DestXSPassword"
export https_proxy=http://proxy.wdf.sap.corp:8080
echo " "
echo " "
echo " "
preversion="$(/home/i847772/sap/hdbclient/hdbalm -s -y -h $bamboo_DestHostname -p $bamboo_DestHostPort -u $bamboo_DestXSUsername -c $bamboo_DestSSLCert du get $bamboo_DeliveryUnitName $bamboo_DeliveryUnitVendor)"
if [[ $preversion == "" ]]; then  echo "Initial install of the DU";  preinstallversion="0.0.0"
else  preinstallversion=$(echo $preversion | grep -Po '(?<=version:)[^r]+' | xargs)
fi
echo "Pre Install version: $preinstallversion"
IMPORT_LOG="$(/home/i847772/sap/hdbclient/hdbalm -s -y -j -h $bamboo_DestHostname -p $bamboo_DestHostPort -u $bamboo_DestXSUsername -c $bamboo_DestSSLCert import "$bamboo_DeliveryUnitFilename")"
postversion="$(/home/i847772/sap/hdbclient/hdbalm -s -y -h $bamboo_DestHostname -p $bamboo_DestHostPort -u $bamboo_DestXSUsername -c $bamboo_DestSSLCert du get $bamboo_DeliveryUnitName $bamboo_DeliveryUnitVendor)"
if [[ $postversion == "" ]]; then  echo "Unable to query installed delivery unit version"  postinstallversion="-1"
else  postinstallversion=$(echo $postversion | grep -Po '(?<=version:)[^r]+' | xargs)
fi
echo "Post Install version: $postinstallversion"
export HDBALM_PASSWD=""
LOG="${IMPORT_LOG##* }"
if grep -q "Successfully imported delivery units" $LOG && [[ $postinstallversion == $preinstallversion ]]; then    echo " "    echo " "    echo "******************************************************* Import of the DU completed, but the version has not changed  *******************************************************"    echo " "    echo "Its possible you have not incremented the version numbers"    echo " "    echo "******************************************************* Log File $LOG *******************************************************"    echo " "    echo " "       if [ $LOG != "" ]; then        cat $LOG    else        echo "No log file, ensure the job is running on a machine with HDBALM"    fi    echo " "    echo " "    echo "******************************************************* //Log File  *****************************************************"    echo " "    echo " "    exit 0   
elif [ $postinstallversion == "-1" ]; then    echo " "    echo " "    echo "******************************************************* Import of the DU Has failed  *******************************************************"    echo " "    echo "******************************************************* Log File *******************************************************"    echo " "    echo " "       if [ $LOG != "" ]; then        cat $LOG    else        echo "No log file, ensure the job is running on a machine with HDBALM"    fi    echo " "    echo " "    echo "******************************************************* //Log File  *****************************************************"    echo " "    echo " "    exit 1    
else    echo " "    echo " "    echo "******************************************************* Import of the DU has completed successfully  *******************************************************"    echo " "    echo "Installation successful"    echo " "    echo " "    exit 0
fi
exit 0

Once our release has been tested on our test environment and are ready to production deployment, we use the deploy function of Bamboo to deploy the Delivery unit to our production instance. This process is identical to test, with the exception that the delivery unit is not pulled from github but rather the the build job artifacts (since the code we want to release, might not be the code in our dev repo du currently). In this case we always specify the version of the Dev to Test build job.

Our deployment to production is (currently) not done by developers,

but our operations team. This is still performed in bamboo.

Bamboo screenshot (just for reference)

Devops on HANA XSA Projects

Since we no longer have the HANA repository on XSA, our build jobs/processes are going to be greatly simplified and very similar to traditional projects on the new platform. (Awesome!)

If you have comments, questions or suggestions we are always interested to hear and share more around these topics.