Thursday, March 2, 2017

Ballerina 101

In this tutorial you will learn the basic concepts of Ballerina and why we call it Flexible, Powerful and Beautiful. You will also learn to run your ballerina programs in two modes: server and standalone modes with simple examples.


Ballerina is a programming language designed from the ground up specifically for integration which allows you to draw code to life. It allows you to connect apps and services to handle all kinds of integration scenarios. Why do we call it Flexible, Powerful and Beautiful?

You can build your integrations by drawing sequence diagrams, or write your code in swagger or in ballerina. You can add plugins and write ballerina code in IntelliJ IDEA[2], Vim[3], Atom[4], Sublime Text 3[5] and more. Therefore it is FLEXIBLE.

Ballerina can handle everything from a simple Hello World program to complex service chaining and content-based routing scenarios. It comes with native support for REST, Swagger, JSON, and XML, and it includes connectors for popular services like Twitter and Facebook. It has an incredibly fast lightweight runtime which can be deployed in a production environment without any development tools. Therefore it is POWERFUL.

The integration code is written for you as you create the diagram in Ballerina Composer.

Cool! isn't it? All you have to do is drag and drop elements needed for your use case onto a canvas who will easily create your integration scenario for you. You can switch between Source view and Design view anytime. Therefore it is BEAUTIFUL.

Key Concepts

Each Ballerina program represents a discrete unit of functionality that performs an integration task. The complexity of the ballerina program is upon your discretion.

You can create your ballerina program in two ways.
1. Server mode: as a service that runs in the Ballerina server and awaits requests over HTTP.
2. Standalone mode: as an executable program that executes a main() function and then exits.

Following are the available constructs you can use [1].

1. Service: When defining a Ballerina program as a service instead of an executable program, the service construct acts as the top-level container that holds all the integration logic and can interact with the rest of the world. Its base path is the context part of the URL that you use when sending requests to the service.

2. Resource: A resource is a single request handler within a service. When you create a service in Ballerina using the visual editor, a default resource is automatically created as well. The resource contains the integration logic.

3. Function: A function is a single operation. Ballerina includes a set of native functions you can call and you can define additional functions within your Ballerina programs.
 The main() function contains the core integration logic when creating an executable program instead of a service. When you run the program, the main() function executes, and then the program terminates. You can define additional functions, connectors, etc. inside the program and call them from main(). See here for a complex example.

4. Worker: A worker is a thread that executes a function.

5. Connector: A connector represents a participant in the integration and is used to interact with an external system or a service you've defined in Ballerina. Ballerina includes a set of standard connectors that allow you to connect to Twitter, Facebook, and more, and you can define additional connectors within your Ballerina programs.

6. Action: An action is an operation you can execute against a connector. It represents a single interaction with a participant of the integration.

See language reference for more information.

Quick Start

1. Download complete ballerina tools package from
2. Unzip it on your computer and lets call it <ballerina_home>
e.g.: /WSO2/ballerina/ballerina-tools-<version>
3. Add <ballerina_home>/bin directory to your $PATH environment variable so that you can run ballerina commands from anywhere.
e.g.: on Mac OS X

export BALLERINA_HOME=/WSO2/ballerina/ballerina-tools-0.8.1

Run HelloWorld - Standalone Mode

Now we are going to run HelloWorld classical example using a main() function, i.e. in standalone mode as follows.

1. Create /WSO2/ballerina/tutorial/helloWorld directory.
2. In this directory, create the file helloWorld.bal with following contents.

import ballerina.lang.system;

function main (string[] args) {
  system:println("Hello, World!");

This is how the famous hello world sample looks like in ballerina programming language!

3. Issue the following command to run the main() function in helloWorld.bal file.
$ ballerina run main helloworld.bal

You can observe the following output in command line.
> Hello, World!

After the HelloWorld program is executed, Ballerina will be stopped. This is useful when you want to execute a program once and then stop as soon as it has finished its job. It runs the main() function of the program you specify and then exits

Run HelloWorld - Server Mode

Here Ballerina will deploy one or more services in the ballerina program that wait for requests.
1. Create the file helloWorldService.bal with following contents.

import ballerina.lang.messages;
@http:BasePath ("/hello")
service helloWorld {

    resource sayHello (message m) {
        message response = {};
        messages:setStringPayload(response, "Hello, World!");
        reply response;



3. Issue the following command to deploy the helloWorld service in helloWorldService.bal file.
$ ballerina run service helloWorldService.bal

You can observe the following output in command line that the service is waiting for requests.
> ballerina: deploying service(s) in 'helloWorldService.bal'
> ballerina: started server connector http-9090

The Ballerina server is available at localhost:9090, and helloWorld service is available at context hello

4. Open another command line window and use the curl client to call helloWorld service as follows.
$ curl -v http://localhost:9090/hello

The service receives the request and executes its logic, printing "Hello, World!" on the command line as follows. 
*   Trying
* Connected to localhost ( port 9090 (#0)
> GET /hello HTTP/1.1
> Host: localhost:9090
> User-Agent: curl/7.49.1
> Accept: */*
< HTTP/1.1 200 OK
< Content-Type: text/plain
< Content-Length: 13
* Connection #0 to host localhost left intact
Hello, World!

Notice that the Ballerina server is still running in the background, waiting for more requests to serve. 

5. Stop the Ballerina server by pressing Ctrl-C (Command-C). 


Tuesday, February 21, 2017

Ballerina is born!

What is ballerina?
What is ballerinalang?

Ballerina - a new open source programming language that lets you 'draw' code to life!

It is a programming language that lets you create integrations with diagrams.

At WSO2, we’ve created a language where diagrams can be directly turned into code. Developers can click and drag the pieces of a diagram together to describe the workings of a program. Cool! isn't it?

We’re not just targeting efficiency, but also a radical new productivity enhancement for any company. By simplifying the entire process, we’re looking at reducing the amount of work that goes into the making of a program. It’s where we believe the world is headed.

As mentioned by Chanaka [4], there is a gap in the integration space where programmers and architects speaks in different languages and sometimes this resulted in huge losses of time and money. Integration has lot to do with diagrams. Top level people always prefer diagrams than code but programmers do the other way around. We thought of filling this gap with a more modernized programming language.

Ballerina features both textual and graphical syntaxes that uniquely offer the exact same expressive capability and are fully reversible. The textual syntax follows the C/Java heritage while also adopting some aspects from Go. The graphical syntax of Ballerina follows a sequence diagram metaphor. There are no weird syntax exceptions, and everything is derived from a few key language concepts. Additionally, Ballerina follows Java and Go to provide a platform-independent programming model that abstracts programmers from machine-specific details.

We are happy to announce the “Flexible, Powerful, Beautiful” programming language “Ballerina”. Here are the main features of the language in a short list [4].
  • Textual, Visual and Swagger representation of your code.
  • Parallel programming made easier with workers and fork-join.
  • XML, JSON and DataTable as built in data types for easier data handling.
  • Packaging and module system to write, share, distribute code in elegant fashion.
  • Composer (editor) makes it easier to write programs in a more visual manner.
  • Built in debugger and test framework (testerina) makes it easier to develop and test.
Ballerina supports high-performance implementations—including the micro-services and micro-integrations increasingly driving digital products—with low latency, low memory and fast start-up. Notably, common integration capabilities are baked into the Ballerina language. These include deep alignment with HTTP, REST, and Swagger; connectors for both web APIs and non-HTTP APIs; and native support for JSON, XML, data tables, and mapping.

Tryout ballerina and let us know your thoughts on medium, twitter, facebook, slack, google and many other channels.

Ask a question in stackoverflow.

Have fun!

Photo credits:

You can find the introduction to Ballerina presentation below presented by Sanjiva at WSO2Con 2017 USA.


Wednesday, October 19, 2016

Where did everything go in Puppet 4.x?

This is a summary of [1] at

  • New All-in-One puppet-agent package - On managed *nix systems, you’ll now install puppet-agent instead of puppet. (This package also provides puppet apply, suitable for standalone Puppet systems.) It includes tools (private versions) like Facter, Hiera, and Ruby; also MCollective.
  • *nix executables are in /opt/puppetlabs/bin/ - On *nix platforms, the main executables moved to /opt/puppetlabs/bin. This means Puppet and related tools aren’t included in your PATH by default. You have to add it to your PATH or use full path when running puppet commands. 
    1. Private bin directories - The executables in /opt/puppetlabs/bin are just the “public” applications that make up Puppet. Private supporting commands like ruby and gem are in /opt/puppetlabs/puppet/bin
  • *nix confdir is Now /etc/puppetlabs/puppet - Puppet’s system confdir (used by root and the puppet user) is now /etc/puppetlabs/puppet, instead of /etc/puppet. Open source Puppet now uses the same confdir as Puppet Enterprise.
    1. ssldir is inside confdir - The default location is in the $confdir/ssl on all platforms.
    2. Other stuff in /etc/puppetlabs - Other configs are in /etc/puppetlabs directory. Puppet Server now uses /etc/puppetlabs/puppetserver, and MCollective uses /etc/puppetlabs/mcollective.
  • New codedir holds all modules(1), manifests(1) and data(2) - The default codedir location is /etc/puppetlabs/code on *nix. It has environments, modules directories and hiera.yaml config file.
    1. Directory environments are always on - The default environmentpath is $codedir/environments and a directory is created at install for the default production environment. Modules are in $codedir/environments/production/modules and main manifest is in $codedir/environments/production/manifests. However you can still use global modules in $codedir/modules and a global manifest.
    2. Hiera data goes in environments by default - Hiera’s default settings now use an environment-specific datadir for the YAML and JSON backends. So the production environment’s default Hiera data directory would be /etc/puppetlabs/code/environments/production/hieradata
  • Some other directories have moved - The system vardir for puppet agent has moved, and is now separate from Puppet Server’s vardir. For *nix: /opt/puppetlabs/puppet/cache. The rundir, where the service PID files are kept, (on *nix) has moved to /var/run/puppetlabs. (Puppet Server has a puppetserver directory in this directory.)


Wednesday, October 12, 2016

Accessing Google APIs using OAuth 2.0

Here is my use case.
I recently wanted to use WSO2 ESB GoogleSpreadsheet Connector. Before attempting any other spreadsheet operation in connector, I have to call the googlespreadsheet.init element first. This configuration authenticates with Google Spreadsheet by configuring the user credentials using OAuth2 authentication for accessing the Google account that contains the spreadsheets.

I couldn't initially get any simple answers/instructions in www on how to set up the following four elements from my Google Account. I have shared the process for your benefit hoping this will save loads of your valuable time.
1. Client ID
2. Client Secret
3. Refresh Token
4. Access Token

1. Go to
2. Under Credentials -> Create Credentials -> OAuth client ID

3. Select Application Type: Web Application
    Name: <appName>
   Authorized redirect URIs:

4. Click Create.
Google will immediately show your Client ID and Client Secret as follows (you can get these details later also).

Following is how the edit view shows you the required information.

6. Click the settings icon.
7. Select the Use your own OAuth credentials check box
8. Fill two fields OAuth Client ID and OAuth Client secret with the values you retrieved in step 4.

9. Now in Step 1 select the Google Sheets APIv4 all 4 scopes as shown below.
10. Click on Authorize APIs
11. Once ask approve permission to access the selected scopes.

 12. In Step 2 Click on Exchange authorization code for tokens button.
13. These are your Refresh and Access Tokens

Now all four fields required are retrieved.
Following is how you will be using googlespreadsheet.init element inside your proxy/sequence configuration.


Following is a sample JSON payload to invoke googlespreadsheet.init method in ESB google spreadsheet connector. Note that the registry path is optional here.

  "googlespreadsheetRefreshToken": "1/KSKKDKDK_Isdf_dsalkfdffafdskfdsafdksfj_sdfSWE",
  "googlespreadsheetClientId": "",
  "googlespreadsheetClientSecret": "0GeUJ4KK3Flk8C9S0Ua51jZS",
  "googlespreadsheetApiUrl": ""

Note: As of today WSO2 ESB googlespreadsheet connector is compatible with Google Spreadsheets API v3(Legacy). But for authentication purposes there will be no much of a difference. I have written my own connector for Google Spreadsheets API v4 to use v4 specific methods like spreadsheets.create
Refer [1] for more information on writing your own connector.


Tuesday, October 11, 2016

Building/Uploading/Using the Sample ESB Connector in few minutes

I have used WSO2 ESB 5.0.0 in this article.
Go to an empty folder and run the following command to generate the maven project sample connector code.

mvn archetype:generate -DarchetypeGroupId=org.wso2.carbon.extension.archetype -DarchetypeArtifactId=org.wso2.carbon.extension.esb.connector-archetype -DarchetypeVersion=2.0.0 -DgroupId=org.wso2.carbon.connector -DartifactId=org.wso2.carbon.connector.sample -Dversion=1.0.0 -DarchetypeRepository=

You will be asked to specify a name for the connector.
Define value for property 'connector_name': : Sample
After asking to confirm the properties type Y and enter to confirm.

Issue the following command to build the connector zip file.

mvn clean install -Dmaven.test.skip=true

Download/Start WSO2 ESB 4.9.0 or ESB 5.0.0 Server.

Login to the management console (default username/password: admin/admin) and upload the created 
connector zip file.
(zip file will be at ../org.wso2.carbon.connector.sample/target/

Add the following proxy service. (Home -> Manage -> Services -> Add -> Proxy Service)
Select custom proxy service -> view source -> paste following code -> Save

<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns=""
         <log level="custom">
            <property name="**********" value="Inside sampleConnectorProxy **********"/>

Following will be printed in the server log file

TID: [-1234] [] [2016-10-11 13:33:36,755]  INFO {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker} -  Proxy service : sampleConnectorProxy was added to the Synapse configuration successfully {org.wso2.carbon.mediation.dependency.mgt.DependencyTracker}
TID: [-1234] [] [2016-10-11 13:33:36,755]  INFO {org.apache.synapse.core.axis2.ProxyService} -  Building Axis service for Proxy service : sampleConnectorProxy {org.apache.synapse.core.axis2.ProxyService}
TID: [-1234] [] [2016-10-11 13:33:36,757]  INFO {org.apache.synapse.core.axis2.ProxyService} -  Adding service sampleConnectorProxy to the Axis2 configuration {org.apache.synapse.core.axis2.ProxyService}
TID: [-1234] [] [2016-10-11 13:33:36,758]  INFO {org.wso2.carbon.core.deployment.DeploymentInterceptor} -  Deploying Axis2 service: sampleConnectorProxy {super-tenant} {org.wso2.carbon.core.deployment.DeploymentInterceptor}
TID: [-1234] [] [2016-10-11 13:33:36,759]  INFO {org.apache.synapse.core.axis2.ProxyService} -  Successfully created the Axis2 service for Proxy service : sampleConnectorProxy {org.apache.synapse.core.axis2.ProxyService}
Go to services -> Try this service -> Send

Following will be printed in the server log file

TID: [-1234] [] [2016-10-11 13:33:47,314]  INFO {org.apache.synapse.mediators.builtin.LogMediator} -  ********** = Inside sampleConnectorProxy ********** {org.apache.synapse.mediators.builtin.LogMediator}
TID: [-1234] [] [2016-10-11 13:33:47,316]  INFO {org.apache.synapse.mediators.builtin.LogMediator} -  To: /services/sampleConnectorProxy.sampleConnectorProxyHttpSoap12Endpoint, WSAction: urn:mediate, SOAPAction: urn:mediate, MessageID: urn:uuid:e3b5f31c-37b8-4ca8-b483-9bbc4a6daf04, Direction: request, *******template_param******** = Hi, Envelope: <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv=""><soapenv:Body/></soapenv:Envelope> {org.apache.synapse.mediators.builtin.LogMediator}
TID: [-1234] [] [2016-10-11 13:33:47,316]  INFO {org.wso2.carbon.connector.sampleConnector} -  sample sample connector received message :Hi {org.wso2.carbon.connector.sampleConnector}

To do modifications to the sample connector and experiment, you can set up your development environment. I'm using Intellij IDEA Community Edition. Refer to the following command.

mvn clean install idea:idea

Learn more about the components inside the connector.

Creating a third party connector and publishing it in WSO2 store.

Wednesday, August 24, 2016

WSO2 DSS, Secure Data Service using Basic Auth

Hope you have read this article: Using WSO2 DSS retrieve data from multiple databases in a single Query. Lets use the same data service and secure it using Basic Auth. Then lets try to invoke the secured data service with SOAP UI or from a WSO2 ESB.

Securing the data service
1. If not already created, add a new collection to /_system/config as security, i.e., /_system/config/security
2. Then add this policy file[1] (if you want you can rename the policy file, say policy1.xml) to /_system/config/security/ using Add Resource | Upload content from file | choose file

3. Next you have to add this policy to your data service. Go to data service xml editor view.
Add following to Data Service configuration at the end just before </data> closing tag.

   <policy key="conf:security/policy1.xml"/>

Wait for few seconds and go to Deployed Services page and view the available data services. Now you can see the updated data service is as secured.

When you are testing the data service from the SOAP UI, you need to send the Authorization header.
Go to request level of your SOAP UI project created in the previous article and add Header as follows.

Header Name : Authorization
Value : Basic YWRtaW46YWRtaW4=

We are using admin:admin default credentials here.
YWRtaW46YWRtaW4= is the base64 encoded value of admin:admin
You can calculate the base64 encoded value online [2].

If a WSO2 ESB is invoking the service simply add following configuration in the synapse config
before sending message to the endpoint.

<property xmlns:ns="http://org.apache.synapse/xsd"
 expression="fn:concat('Basic ', base64Encode('username:password'))"

If you are using jaggery to call DSS endpoints you can send the headers with the request as follows.

var POST_HEADERS = { "Content-Type": "application/json", "Authorization": "Basic YWRtaW46YWRtaW4="};

var resp = put(dssLDAPUserDSURL + "/employee/status", stringify(dataPost), POST_HEADERS);

You can find the sample data service, policy file, SOAP UI project and the relevant MySQL database scripts here [3].


Tuesday, August 23, 2016

Using WSO2 DSS retrieve data from multiple databases in a single Query

I came across an issue of having same table in multiple databases and having to write data multiple times when syncing these data with external systems. Therefore I thought of transferring these duplicated tables into one single common database. Then this common database will to be used by several of our internal systems. However I recently came across an issue that I needed to retrieve data from multiple databases using a single query. Since I'm already using WSO2 DSS, I found a solution to tackle this problem.

Here's how to do it. For the ease of understanding I have divided the steps into two parts.

Part 1 - Configure MySQL Database

Login to your local MySQL db and create two databases.
Then create two tables and insert some test data as follows.

mysql> create database db1;
mysql> create database db2;

mysql> use db1; 
mysql> CREATE TABLE employees ( EmployeeID int(11) NOT NULL AUTO_INCREMENT, FirstName varchar(255) DEFAULT NULL, LastName varchar(255) DEFAULT NULL, Team varchar(255) DEFAULT NULL, PRIMARY KEY (EmployeeID));
mysql> insert into employees (FirstName, LastName, Team) values('Suhan', 'Dharmasuriya', 'InternalIT');
mysql> insert into employees (FirstName, LastName, Team) values('Thilina', 'Perera', 'Finance');
mysql> select * from employees;
| EmployeeID | FirstName | LastName     | Team       |
|          1 | Suhan     | Dharmasuriya | InternalIT |
|          2 | Thilina   | Perera       | Finance    |
2 rows in set (0.00 sec)

mysql> use db2;
mysql> CREATE TABLE engagements ( EngagementID int(11) NOT NULL AUTO_INCREMENT, EmployeeID int(11), PRIMARY KEY (EngagementID));
mysql> insert into engagements (EmployeeID) values(1);
mysql> select * from engagements;
| EngagementID | EmployeeID |
|            1 |          1 |
1 row in set (0.00 sec)

Now lets test the following MySQL query and get results. Here I have used a LEFT OUTER JOIN.
Result will be obtained from the two databases. We will save this query in WSO2 DSS. 

mysql> SELECT A.EmployeeID, A.FirstName, A.LastName, A.Team, B.EngagementID from db1.employees AS A LEFT OUTER JOIN db2.engagements AS B ON A.EmployeeID=B.EmployeeID;
| EmployeeID | FirstName | LastName     | Team       | EngagementID |
|          1 | Suhan     | Dharmasuriya | InternalIT |            1 |
|          2 | Thilina   | Perera       | Finance    |         NULL |
2 rows in set (0.00 sec)

Part 2 - Configure WSO2 DSS

  1. Download WSO2 DSS 3.5.0 from here ( If you already have the product zip file ( continue with the next step.
  2. Unzip the product to a path containing no spaces in the path name. This is your <DSS_HOME>
  3. Download the MySQL connector jar here ( and copy it to <DSS_HOME>/repository/components/lib/
  4. Start the WSO2 DSS server.
To start the server, your have to run the script wso2server.bat (on Windows) or (on Linux/Solaris) from the <DSS_HOME>/bin folder.
  1. Log in to DSS by using the default credentials (username: admin/ password: admin).

Creating the data service

  1. Create a new data service 'sampleDS'
  2. Home -> Manage -> Services -> Add -> Data Service -> Create
  3. Create a datasource as follows by referring to the above local MySQL database.
  4. Add new Query; query ID: 'getEngagementsPerEmployee', query: above tested query, and press generate response link
  5. No operation is defined in this example, therefore press next
  6. Add new Resource; Resource Path: '/engagements' by selecting Resource Method: 'GET' and selecting Query ID: 'getEngagementsPerEmployee'
  7. Press Finish to create the data service

Refer following screenshots for more information.

Now you can test the datasource as follows.

Open a new tab in your browser and type the following URL: http://localhost:9763/services/sampleDS/engagements

You will get the following response:
<Entries xmlns="">
<EngagementID xmlns:xsi="" xsi:nil="true"/>

Refer following Data Service synapse configuration for more information.
<data name="sampleDS" transports="http https local">
   <config enableOData="false" id="datasource">
      <property name="driverClassName">com.mysql.jdbc.Driver</property>
      <property name="url">jdbc:mysql://localhost:3306</property>
      <property name="username">root</property>
      <property name="password">root</property>
   <query id="getEngagementsPerEmployee" useConfig="datasource">
      <sql>SELECT A.EmployeeID, A.FirstName, A.LastName, A.Team, B.EngagementID from db1.employees AS A LEFT OUTER JOIN db2.engagements AS B ON A.EmployeeID=B.EmployeeID;</sql>
      <result element="Entries" rowName="Entry">
         <element column="EmployeeID" name="EmployeeID" xsdType="string"/>
         <element column="FirstName" name="FirstName" xsdType="string"/>
         <element column="LastName" name="LastName" xsdType="string"/>
         <element column="Team" name="Team" xsdType="string"/>
         <element column="EngagementID" name="EngagementID" xsdType="string"/>
   <resource method="GET" path="/engagements">
      <call-query href="getEngagementsPerEmployee"/>