Saturday, July 30, 2016

Caltech Smart Charging EV Network with IoT Analytics

We had the opportunity to work with Caltech recently on analytics for their smart EV charge grid. The following piece is a glimpse into their Smart Charging EV technology that has the potential to transform EV charging networks across parking structures.

While electric vehicles are poised to breakout into the mainstream, the lack of an ubiquitous charging infrastructure remains an impediment to the ease and accessibility of EV cars.

Caltech has developed an Adaptive Charging Network that allows garages to significantly expand their EV footprint, without expensive electric grid upgrades.

First, let’s look at the state of play within the EV market.

By the Numbers

Currently, there are nearly 500,000 highway capable plug-in electric vehicles in the United States. California is the bastion of Electric Vehicles sales in the U.S., accounting for over half of all plugin electric vehicles sold in the United States, and 3.1% of all vehicles sold within the state.

Tesla unveiled its Model 3 amid much fanfare earlier in the year. A whopping 373,000 pre-orders shortly thereafter, Tesla has accelerated its production plan of 500k cars per year in 2018, two years ahead of their own long standing plan. All major car manufactures are also getting in the act, with billions of dollars in EV investment from the likes of BMW, Mercedes Benz, GM, Ford and others.

The Obama administration has also announced plans to accelerate EV adoption in the United States, with upto $4.5 billion in loan guarantees.

So, all signs indicate that we are at the cusp of EV adoption breaking into the mainstream. For EVs to be mass market, however, consumers need convenient and affordable charging infrastructure to be on par with gasoline powered vehicles of today.

The Infrastructure Problem

EV charging takes a significant amount of electricity, to the tune of 70 desktop computers simultaneously for a single EV charge.

Parking lots are a logical place for EV charging networks, particularly for day-to-day, commute → park scenarios. However, parking structures are not well equipped for a large number of EV stations, as it requires expensive upgrades to the electricity infrastructure. As a result you’ll typically find only a handful of EV ports in supported parking structures.

What if garages were able to offer large number of EV slots, without requiring electric grid updates, and interchangeably use it for general parking for gasoline powered cars?

Enter Caltech’s Adaptive Charging Network.

Adaptive Charging Network
Caltech’s Adaptive Charging Network, built by the team lead by Professor Steven Low, uses a smart algorithm that coordinates the charging schedule with the electrical infrastructure.

Based on a variety of factors including spots occupied, vehicle amps, charged vehicles, number of hours required for each vehicle and other factors, it determines the optimum charge associated to each spot.

The system is currently used within the California campus, providing an unprecedented 54 charging stations, with minimal impact to their electrical infrastructure.

Each charging station has an ultrasonic sensor that transmits the status of the parking spot. This opens up the spots for general parking while also leaving dedicated slots for EV charging based on demand.

Currently, the stations consume 200 Kilo Watt Hours/day, a small fraction of (0.00006%) of Caltech’s overall energy usage. 30% of electricity per station comes from carbon free renewable sources.
Two transformers with an overall capacity of 300KW handles the load with charge dynamically allocated to vehicles based upon need.

A touch screen system allows users to define their vehicle type, length of parking time as well as miles required to reach their destination.

Charge Analytics

The system collects a range of sensor data from each of its charging stations and stored into NoSQL databases. Using Cloud9 Charts, this data is distilled to determine realtime availability of parking spots, and whether the occupied stations are currently charging or done charging, as well as displayed via real-time dashboards on TV screens across the campus.

“Range Anxiety” is one of the primary barriers for mass market EV adoption. Charging infrastructure must be readily available before consumers fully embrace the EV movement.

Innovative solutions like Caltech’s Adaptive Charging Network, along with dedicated charging stations brings us a step closer towards easily accessible, convenient and cost-effective charging infrastructure.


Tuesday, July 5, 2016

Review: MongoDB Atlas Database-as-a-Service


The big news at MongoDB World 2016 last week was the launch of Atlas, MongoDB's own Database-as-a-Service (DBaaS). Atlas makes deployment and management of MongoDB easier, with managed hardware provisioning, failure recovery and backups.

Atlas is currently available on AWS, with the service soon available within Azure and Google Platform.


For MongoDB, Atlas is a logical entry into the DBaaS world, as it focuses on additional revenue streams from its open source model.

With this move, MongoDB is encroaching into the domain of current MongoDB DBaaS players, most notably mLab and (acquired by IBM last year). There's little doubt that there's potentially a significant market for Atlas: MongoDB has been downloaded over 10M times; there are over 300K hosted deployments on mLab.

So does this spell doom and gloom for current MongoDB DBaaS vendors? Not so fast.

While we expect Atlas to eventually be the leader within the MongoDB DBaaS market, current market leaders have already aligned their strategy to support "polyglot" persistence models. Polyglot persistence implies using the right database for the right workload. For example, an organization may use relational databases for structured data, ElasticSearch for search, Redis for caching, Cassandra for high volume writes, MongoDB or Couchbase for semi-structured data, etc.

We believe that this is a wise strategy on their part, as NoSQL vendors will logically be locked into selling their own DBaaS. In our own experience, polyglot database architectures are quickly becoming the norm rather than the exception. For DevOps in a polyglot environment, managing a single platform like is more appealing for than managing multiple DBaaS providers.



Building a cluster in Atlas was delightfully straightforward: simply specify the AWS region, instance size, storage and a few other things like encryption and payment details to provision your cluster.

Provisioning took approximately 10 minutes for a 3 node cluster.   


Atlas paid plans are hourly and currently starts at $56/month based on the size, storage and nodes.

There are no free/sandbox environments within Atlas. 2017 UPDATE: MongoDB Atlas now offers a free sandbox environment. Sign up free here. Alternative environments to kick the tires, are offered from mLab.

Paid plans at mLab start at $15/mo and $31/mo for Pricing plans differ across all three, so we recommend determining your own usage needs to compare pricing across all three based on your own requirements.

Atlas pricing is clearly displayed during the provisioning process. Atlas also presents the monthly costs incurred transparently, on the dashboard updated on a daily basis.


After the nodes are provisioned, Atlas will prompt you to setup IP whitelists for outside connectivity into the cluster. This was also a painless process, with the ability to add IPs to the whitelist, being able to download client drivers as well the shell all in one neatly packaged dialog.

Atlas enables you to create database users via the 'Security' tab.  However, creating users against specific databases can be a bit confusing. Users created will be defaulted to the admin authentication database (we were not able to find a way to change this within Atlas). So, when you try to connect to your own database, remember to pass in admin as the authentication database parameter into the connect string.

> mongo mongodb://,, --ssl -authenticationDatabase=admin --username <user> --password <pass>


By default, the clusters are empty without any sample data. In addition, our client download did not contain mongoimport utility as part of the download; it was only enabled us to connect to the cluster.

Here's what we did:
  a) Downloaded a sample JSON dataset.
  b) Used MongoHub to import the JSON file into a collection.
It would have been nice if Atlas provided the ability to seamlessly import test databases directly from the portal.


There are no query facilities within Atlas, so you'll need mongo shell or a third party client application to peek into the data after import.

See Part 2 of this blog around visualization and analysis of data hosted in Atlas (using Cloud9 Charts).


We found the operational aspects of Atlas to be quite comprehensive:

Each node in the main dashboard opens up a variety of operational metrics for the node, with realtime database, hardware and other stats as well as the ability to customize the metrics displayed.

A configurable set of operational alerts, with the ability to define groups, roles and conditions upon which alerts can be raised.

Backups are automatically managed, with a configurable schedule on when snapshots are taken.


We found Atlas to be easy to use with painless cluster deployment and management. We were also quite impressed by the operational aspects of the solution. Database setup and user management needs some refinements, but overall, it looks like MongoDB might have a winner in its hands.

Jay Gopalakrishnan is the Founder of Cloud9 Charts, the leading native analytics platform on NoSQL databases.

Read on for analysis and visualizations natively on a hosted Atlas deployment.

Friday, July 1, 2016

Native Analytics on Couchbase 4.5

Couchbase 4.5 - Native BI Support

This originally appeared as a guest blog on Couchbase.

Earlier this week, Couchbase launched the latest version of its flagship NoSQL database, 4.5, with powerful additions to N1QL, faster queries and indexing, role based security and a range of other features.  

At Knowi, we are pleased to add support for 4.5 to bring native Business Intelligence (BI) integration into Couchbase, leveraging the latest N1QL 4.5 features while maintaining backward compatibility.    

The 4.5 release brings forth a number of query related improvements that are particularly useful from a BI & reporting perspective:

  • A new INFER command to determine fields and associated meta data
  • High performance JOINs with Index-JOIN support
  • Array indexing, that boosts the performance of queries involving arrays
  • Memory-optimized indexes that improves query performance under write-intensive workloads
The new INFER command provides metadata of the document structure, as well as providing a sneak-peek of values within it.

For example, running INFER database in the Couchbase Query Console returns the document structure, fields, type and sample field data:

We utilize the INFER command to determine the fields displayed for dynamic N1QL query generation (for versions prior to 4.5, we fall back to querying the first 100 documents to build the field suggestions).

See this action in few simple steps:

  1. Go to This is pre-populated with a live Couchbase database with travel-sample and beer-sample buckets to easily follow along.

  1. Expand Query Generation section. Choose travel-sample. This triggers the INFER command to determine the fields to populate Metrics, Dimensions, Filters and Sort fields.
  1. Use the dropdowns to auto-generate N1QL queries using a point and click interface to build a simple count aggregation by the type field as below.

4. Click on Show me to instantly visualize and see results.

N1QL, with SQL like familiarity while fully supporting JSON, is arguably the best query syntax that we’ve worked with, inside the NoSQL ecosystem. If you are using Couchbase or planning to use Couchbase, leverage N1QL to it’s fullest using Knowi, the only fully native end-to-end BI solution on Couchbase.