Wednesday, May 15, 2019

Integrate WSO2 APIM with ETCD

Overview  


Wide adoption of SOA/MSA influenced modern systems to be disaggregated. These disparate systems need to be integrated together to form a business unit or an application. These systems as a whole are being implemented based on well defined standards/interfaces i.e http, jms which makes it possible to seamlessly integrate. Each interface introduces a way to locate these systems/resources i.e url in the form of an endpoint. These endpoints could change rapidly, i.e ip change etc, which creates a requirement to centrally manage them and to allow services/systems who are connecting with each other to discover them.

The following describes how WSO2 APIM could be integrated with ETCD to dynamically discover the endpoints. The tutorial depicts the following,

  1. A sample service would represent a unit of work which will be done through a warehouse management system.
  2. The services are exposed as rest endpoints, and they could be accessed through the ip and port (localhost, 8090)
  3. Hypothetically let’s consider that the port which the service is running would change from time to time.
  4. Each time the port changes it would not be practical to reflect the change in endpoint to each other service/application which consumes it

Hence, following would demonstrate how the service url could be added as an entry to etcd registry, this entry would be an identifier which will represent the url of the service. Connecting applications will query for the url of this endpoint through the registry instead of directly having to embed the url in the application. The change in the url endpoint should only be reflected in the registry entry. The change will be reflected to the connecting applications without having to apply any changes to the application.

Sample Service


Make sure that Ballerina is downloaded and configured. For detailed instructions please refer here.

A sample mock service could be found here.

Once the service is available, execute the following,



ballerina run abc_warehouse.bal

The above will start the service in localhost:8090, the following message would be prompted in the console as a result,


Initiating service(s) in 'abc_warehouse.bal'
[ballerina/http] started HTTP/WS endpoint 0.0.0.0:8090




ETCD Setup


Make sure that ETCD is running, execute the following



The following response could be observed,

{"etcdserver":"3.3.13","etcdcluster":"3.3.0"}

Insert key into the etcd registry/this could also be used to change the entry

curl http://127.0.0.1:2379/v2/keys/warehouseOrderEP -X PUT  -d value="http://localhost:8090/warehouse/order"

To retrieve the key execute the following,





WSO2 APIM Integration


WSO2 APIM provides the capability to define dynamic endpoints. Using this capability in conjunction with mediation extensions. APIM gateway could be integrated with ETCD registry to dynamically discover the service endpoints. 

The following would be the mediation extension which should be added.



<sequence name="warehouseSvcDiscoverySeq" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
  <log>
    <property name="State" value="Discovery Sequence"/>
  </log>
  <call blocking="true">
   <endpoint>
      <http method="get" uri-template="http://127.0.0.1:2379/v2/keys/warehouseOrderEP"/>
    </endpoint>
  </call>
  <log level="full" />
  <property description="WarehouseOrderEP Value" expression="json-eval($.node.value)" name="WarehouseOrderEP" scope="default" type="STRING"/>
  <log>
    <property name="Discovery Endpoint Value" expression="$ctx:WarehouseOrderEP"/>
  </log>
  <header name="To" expression="$ctx:WarehouseOrderEP"/>
</sequence>

The relevant extension could be downloaded from here.

Testing


After applying the above extension. Invoke the WSO2 APIM gateway endpoint.

curl -k -H "Content-Type: application/json" -X POST -d ‘{"cust_id":"hmart", "delivery":"addr1", "contact":"784-7948754", "item":"A100", "quantity":120}’
http://10.100.0.4:8281/warehouse -v
In the warehouse ballerina service following could be observed,

ABC - Order received

2019-05-15 11:40:20,773 INFO [] - Order placed for Item: A100, quantity: 120. Deliverary to: addr1. Contact: 784-7948754

Change the port in which the ballerina service is running (from 8090 to 9090)

In the ETCD registry entry reflect the port change by issuing the following command,

curl http://127.0.0.1:2379/v2/keys/warehouseOrderEP -X PUT -d value="http://localhost:9090/warehouse/order"

After reflecting the change, invoke the APIM gateway URL. The response should be received.

Recommendations


Follow this post to integrate with Consul.

What’s coming up ?


The tutorial explains how ETCD integration could be done via dynamic endpoints and mediation extensions. WSO2 APIM 3.0.0 release would provide out of the box support for this integration with its Micro Gateway offering, for more information refer here.