PromDash metrics

It has been a while,  sorry for that.

I had some memory problems on my 512Mb Digitalocean droplet, so I wanted to analyze the memory consumption.  Digitalocean recently published a tutorial to install Prometheus monitoring system.

Here you can find all the steps to take to stepup Prometheus together with PromDash for custom dashboards.

After that, i added metrics as an A-record to the DNS-management of my purchased domain.

Next, I added a vhost to my apache.conf to point my new domain to the PromDash webserver:

<VirtualHost metrics.sandervl.be:80>
    ServerName metrics.sandervl.be
    ProxyPass / http://localhost:3000/
    ProxyPassReverse / http://localhost:3000/
    <Location />
        AuthType Basic
        AuthName "Restricted Content"
        AuthUserFile /etc/apache2/.htpasswd
        Require valid-user
    </Location>
</VirtualHost>

Off couse,  I wanted to setup some authentication before accessing the PromDash metrics.  Seting up basic authentication with Apache was easy following yet another tutorial from digitalocean:  https://www.digitalocean.com/community/tutorials/how-to-set-up-password-authentication-with-apache-on-ubuntu-14-04

Finally, my system was ready to be monitored.

Next up,  I want do some research setting up email notifications whenever some treshold is passed.  Before that, I have to setup my emailserver.

Stay tuned…

Google Calendar events in Angular js slider

The goal of this post is to create an event slider, where the events come from your Google Calendar using the Google Calendar API.

The first challenge you need to face is loading your javascript libraries in the correct order.  Using the API of Google can be done via the API client library.  Bootstrapping your Angular app can only take place when the API is loaded.  You can achieve this via the callback of the client library

When you use a Google API, you have to generate an API key for public API access at https://console.developers.google.com.  Make sure you enable the Google Calendar API.

<script>
    var googleOnLoadCallback = function() {
            var apisToLoad = 1; // must match number of calls to gapi.client.load()
            var gCallback = function () {
                if (--apisToLoad == 0) {
                    //Manual bootstraping of the application
                    var $injector = angular.bootstrap(document, ['app']);
                }
                ;
            };
            gapi.client.setApiKey('XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX');
            gapi.client.load('calendar', 'v3', gCallback);
    };
</script>
<script type="text/javascript" src="https://apis.google.com/js/client.js?onload=googleOnLoadCallback"></script>

First, we create an Angular service to query the Google Calendar API and fetch the details of your calendar events.  You do this with a factory that exposes a method which returns a promise resolving your events.  As you can see, we can pass in multiple calendarIds to show events or appointments coming from different calendars. The calendar id is an email address that can be found in the address section of your Google Calendar settings.

Secondly,  you create a directive that uses the factory and places the events on the directives scope.  As you can see, I’m using Moment.js  to convert the dates to the format I want.

angular.module('app').directive('eventSlider',['CalendarService',function(CalendarService){
    return {
        restrict: 'A',
        templateUrl: 'templates/event-slider.tpl.html',
        link: function(scope, element, attrs){
            scope.calendar = {};
            var calendarIds = [];
            if(attrs.ids){
                calendarIds=scope.$eval(attrs.ids);
            }
            if(attrs.id){
                calendarIds.push(attrs.id);
            }

            CalendarService.getFutureEventsFromGroups(calendarIds).then(function(events) {
                scope.calendar.events = events;
            });
            scope.calendar.currentEvent=0;
            scope.prevEvent = function(){
                var length = scope.calendar.events.length;
                scope.calendar.currentEvent = ((scope.calendar.currentEvent-1)% length+length)%length;
            };
            scope.nextEvent = function(){
                scope.calendar.currentEvent = (scope.calendar.currentEvent+1)%scope.calendar.events.length;
            };
        }
    }
}]);

angular.module("app").factory('CalendarService', ['$q', function ($q) {
    this.getCalendarEvents = function (calendarId) {
        return gapi.client.calendar.events.list({
            calendarId: calendarId,
            timeMin: moment().toJSON()
        })
    };
    var _self = this;
    return {
        getFutureEventsFromGroups: function (calendarIds) {
            var events = [];
            var deferred = $q.defer();
            var responses = 0;
            angular.forEach(calendarIds, function (calendarId) {
                _self.getCalendarEvents(calendarId, $q).then(function (data) {
                    responses++;
                    var newEvents = data.result.items;
                    for (var i = 0; i < newEvents.length; i++) {
                        if (newEvents[i].start && newEvents[i].start.dateTime) {
                            newEvents[i].start.dateTime = moment(newEvents[i].start.dateTime).toDate();
                        }
                        if (newEvents[i].end && newEvents[i].end.dateTime) {
                            newEvents[i].end.dateTime = moment(newEvents[i].end.dateTime).toDate();
                        }
                        events.push(newEvents[i]);
                    }
                    ;
                    if (responses == calendarIds.length) {
                        return deferred.resolve(events);
                    }
                });
            });
            return deferred.promise;
        }
    }
}]);

Then, you need to create a template to show the events in a slider.  The styling of the slider uses Bootstrap and Font-Awesome.

<div class="calendar">
    <h2>Upcoming activities</h2>

    <div class="panel" ng-repeat="event in calendar.events">
        <div class="eventWrapper" ng-show="$index==calendar.currentEvent">
            <div class="nav nav-left pull-left"><a ng-click="prevEvent()"><i class="fa fa-chevron-circle-left fa-2x"></i></a></div>
            <div class="nav nav-right pull-right"><a ng-click="nextEvent()"><i class="fa fa-chevron-circle-right fa-2x"></i></a></div>
            <div class="wrapper">
                <h3>{{event.summary}}</h3>
                <img class="left png" ng-src="{{event.image}}" alt="logo" ng-if="event.image"/>

                <p class="description">{{event.description}}</p><br/>
                <span class="date">fromn: {{event.start.date}}</span>
                <span class="date">To: {{event.end.date}}</span>
            </div>
        </div>
    </div>
</div>

Finally, you can use your new directive wherever you want like so:

<!-- calendarIds is an array of Google Caledar ids -->
<div class="col-lg-12" event-slider data-ids="{{calendarIds}}"/>
<!-- calendarId is the string version of a Google Calendar id -->
<div class="col-lg-12" event-slider data-id="{{calendarId}}"/>

Good luck!

One-click Deploy to Tomcat

To easily deploy new versions on my Tomcat 7 instance, I wrote some ant-scripts that allowed me to update my application with one push on the button. The application I am talking about can be foud at http://www.sandervl.be/kavvv. This is a website, deployed on Tomcat, consisting of 2 contexts. One is a static resources which contains an Angular JS app.  The other is a WAR-file bundling a Spring application that exploits an API used by the Angular app.

Before you start,  generate some SSH keys on your server, copy the private key to the machine which will run the script and copy the public key in the authorized_keys of your server.  Detailed information can be found on the DigitalOcean docs.

Creating deploy script

At first,  create two tasks in a file deploy_ssh.xml.  One task,  deploy.sandervl.front, unzips an archive to your static resource folder in the webapps directory.  The other tasks,  deploy.sandervl.basket, deploys a WAR-file via the Tomcat manager using  the deploy and undeploy tasks that can be found in the org.apache.catalina.ant-package.  This package resides in catalina-ant.jar, which can be found in the lib folder of your Tomcat installation.

<?xml version="1.0"?>
<project name="Execute Deploy" basedir=".">

    <taskdef name="undeploy" classname="org.apache.catalina.ant.UndeployTask"/>
    <taskdef name="deploy" classname="org.apache.catalina.ant.DeployTask"/>

    <property name="environment" value="production"/>
    <fail unless="environment">environment not set</fail>
    <available file="${basedir}/${environment}.properties" property="properties.present"/>
    <fail unless="properties.present">properties file not found: ${environment}.properties</fail>
    <property file="${basedir}/${environment}.properties"/>

    <target name="deploy.sandervl.front">
        <unzip src="${basedir}/static.zip" dest="/var/www/webapps/front"/>
    </target>

    <target name="deploy.sandervl.basket">
        <deploy-war dns="${host.ip}:${host.port}" package="basket" path="/basket" />
    </target>

    <macrodef name="deploy-war">
        <attribute name="dns" />
        <attribute name="package" />
        <attribute name="path" default="/"/>
        <sequential>
            <undeploy url="http://@{dns}/manager/text" username="${tomcat.username}" password="${tomcat.password}"
                      path="@{path}" failonerror="false"/>

            <deploy url="http://@{dns}/manager/text" username="${tomcat.username}" password="${tomcat.password}"
                    path="@{path}" war="file:${basedir}/@{package}.war"/>

            <delete file="${basedir}/@{package}.war"/>
        </sequential>
    </macrodef>

</project>

Deploy the static resources (deploy.basket.front)

The first step is zipping the static folder.  Exclude unwanted files and folders from the fileset to zip.  Then,  copy your archive together with the deploy_ssh.xml file.  Finally, execute the deploy.sandervl.front task on the server via remote SSH.    Important,  the owner of the webapps/front directory must be the same as the SSH-ser so unzipping the archive does not cause any permission issues.

Deploy the WAR-file (deploy.basket.api)

Copy the WAR-file to your server via SCP.  Again,  also copy the deploy_ssh.xml file and a properties file that contains your Tomcat username and password to log into the manager.  Then,  remotely executes the deploy.sandervl.basket-task defined in deploy_ssh.xml via sshexec.

This blog from Sourcecraft explains how to include the required JAR-files in the ant classpath to execute the deploy and undeploy tasks.  Also,  you will need to copy tools.jar which can be found inside your JDK folder, to /usr/lib/jvm/java-7-openjdk-amd64/lib.  This is needed to load properties from an external file.

<?xml version="1.0"?>
<project name="Execute Deploy" basedir="../">

    <property name="environment" value="production"/>
    <fail unless="environment">environment not set</fail>
    <available file="${basedir}/scripts/${environment}.properties" property="properties.present"/>
    <fail unless="properties.present">properties file not found: ${environment}.properties</fail>
    <property file="${basedir}/scripts/${environment}.properties"/>

    <path id="classpath">
        <fileset dir="${basedir}/scripts" includes="**/*.jar"/>
    </path>

    <target name="deploy.basket">
        <antcall target="deploy.basket.front"/>
        <antcall target="deploy.basket.api"/>
    </target>

    <target name="deploy.basket.front">
        <antcall target="zip.front"/>
        <antcall target="prepare.deploy.sandervl.front"/>
        <antcall target="deploy.sandervl.front.remote"/>
    </target>

    <target name="zip.front" description="Puts front content zip for deploy in production">
        <zip destfile="${basedir}/basket.front/target/static.zip">
            <fileset dir="${basedir}/basket.front/app">
                <include name="**/*"/>
                <exclude name="pom.xml"/>
                <exclude name="static.iml"/>
                <exclude name="**/.sass-cache/**"/>
                <exclude name="**/scss/**"/>
                <exclude name="**/target/**"/>
                <exclude name="**/dummies/**"/>
                <exclude name="**/translations/**"/>
            </fileset>
        </zip>
    </target>

    <target name="prepare.deploy.sandervl.front" description="Puts front content zip for deploy in production">
        <scp todir="${ssh.username}@${host.ip}:${deploy.dir}" keyfile="${ssh.keyfile}" passphrase="${ssh.passphrase}" trust="yes">
            <fileset dir="${basedir}/basket.front/target">
                <include name="static.zip"/>
            </fileset>
            <fileset dir="${basedir}/scripts">
                <include name="deploy_ssh.xml"/>
                <include name="production.properties"/>
            </fileset>
        </scp>
    </target>

    <target name="deploy.sandervl.front.remote">
        <deploy-remote dns="${host.ip}" target="deploy.sandervl.front"/>
    </target>

    <target name="deploy.basket.api">
        <antcall target="prepare.deploy.sandervl.basket"/>
        <antcall target="deploy.sandervl.basket.remote"/>
    </target>

    <target name="prepare.deploy.sandervl.basket" description="Copy basket war to production">
        <upload-war dir="${basedir}/basket/target" dns="${host.ip}" package="basket.war"/>
    </target>

    <target name="deploy.sandervl.basket.remote">
        <deploy-remote dns="${host.ip}" target="deploy.sandervl.basket"/>
    </target>

    <macrodef name="deploy-remote">
        <attribute name="dns"/>
        <attribute name="target"/>
        <sequential>
            <property name="uploadDir" value="${deploy.dir}"/>
            <sshexec host="@{dns}" username="${ssh.username}" passphrase="${ssh.passphrase}" trust="yes"
                     keyfile="${ssh.keyfile}" timeout="300000"
                     command="cd ${uploadDir} &amp;&amp;
    		                  ant -buildfile deploy_ssh.xml @{target}
			                  "/>
        </sequential>
    </macrodef>

    <macrodef name="upload-war">
        <attribute name="dir"/>
        <attribute name="dns"/>
        <attribute name="package"/>
        <sequential>

            <property name="uploadDir" value="${deploy.dir}"/>
            <echo> About to upload package @{dir}/@{package} to ${ssh.username}@@@{dns}:${uploadDir} </echo>
            <sshexec host="@{dns}" username="${ssh.username}" passphrase="${ssh.passphrase}" trust="yes"
                     keyfile="${ssh.keyfile}" timeout="90000" command="mkdir -p ${uploadDir}"/>

            <scp todir="${ssh.username}@@@{dns}:${uploadDir}" keyfile="${ssh.keyfile}" passphrase="${ssh.passphrase}"
                 trust="yes">
                <fileset dir="@{dir}">
                    <include name="@{package}"/>
                </fileset>
                <fileset dir="${basedir}/scripts">
                    <include name="deploy_ssh.xml"/>
                    <include name="production.properties"/>
                </fileset>
            </scp>

        </sequential>
    </macrodef>

</project>

Digitalocean with Apache and Tomcat

A new month, lets write my second post!

I would like to talk about the infrastructural setup and hosting of my own website.  As a hosting provider, I chose to cooperate with DigitalOcean.  I wanted a VPS which I had fully control off, hosted in the cloud to take advantage of the pay-per-use model.  To start with,  I picked the 512 MB droplet with Ubuntu 14.04 installed.

Later on, my choice of provider turned out exactly what I wanted.  The main reason for this was the vast amount of documentation available at the DigitalOcean website. I could easily set up WordPress and Tomcat 7  using the documentation.  Btw, the first results that popped up with a Google search, led me to the DigitalOcean website.

Apache, by default,  run on port 80, Tomcat 7 on 8080.  To set up Apache with port forwarding, I had to configure a virtual host with a proxypass and proxypassreverse to redirect to the Tomcat contexts like so:

ServerName sandervl.be

<VirtualHost *:80>
ServerName sandervl.be
ServerAlias *.sandervl.be
ProxyRequests off

ProxyPass /kavvv http://sandervl.be:8080/front
ProxyPass /basket http://sandervl.be:8080/basket
ProxyPassReverse /kavvv http://sandervl.be:8080/front
ProxyPassReverse /basket http://sandervl.be:8080/basket
</VirtualHost>

My Tomcat instance had one host with two contexts to serve my API (/basket) and static front-end (/front):

<Connector port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
URIEncoding="UTF-8"
redirectPort="8443" />

<Host name="localhost" appBase="/var/www/webapps/basket" unpackWARS="true" autoDeploy="false" xmlValidation="false" xmlNamespaceAware="false">

<Valve className="org.apache.catalina.valves.AccessLogValve" directory="/var/www/logs"
prefix="localhost.access_log." suffix=".txt" pattern="%h %l %u %t &quot;%r&quot; %s %b" resolveHosts="false"/>

<Context path="/front" privileged="false" docBase="/var/www/webapps/front">
</Context>
</Host>

This configuration, together with my low-memory droplet,  caused my Tomcat to crash because it ran out of memory.  Two nice references that helped me a lot to improve some of these performance issues were the following:  enabling swap memory and improving Tomcat start up. Strange but true,  both sources of information come from DigitalOcean community itself.  Coincidence or not,  I am pleased to work with DigitalOcean.

Stay tuned for more!

Hello world!

And we are live!  Finally I decided to launch my own space on the World Wide Web.

This blog will contain some of my ideas and the actual resolutions of some.  Do not expect detailed posts with elaborated tips and tricks, but rather to-the-point and helpful software solutions.  Also,  whenever a new project is launched,  it will be presented over here. Everything will be made publicly available and free to use.

When I say software solutions,  read websites/web applications.  Currently, I am just at the start of my carreer, learning each and every day.  Whatever I learn and seems useful will be shared over here.

This blog will discuss both backend- and front-end development.  Some keyword will be Java, Spring, C#, Umbraco, AngularJs, NodeJs, Bootstrap … Off course,  this can change in the future.

Stay tuned for more!