Why Clustering your Google Map markers in the server side?

Currently, programmers have to face more frequently the development of applications that interact with maps, coordinates and markers. When developing such an application or module, a very important factor must be taken into consideration: performance. Imagine a module that shows a map and that the information generated by the markers grows daily, naturally if the markers increase, more information is sent from the server to the client and the time may come when the application does not respond. The solution will be to group the markers in a cluster to improve the performance in the representation of the information on the map. This solution is not new, there are libraries such as Google Maps API that handle this problem, but they do it completely on the client side asking the server for all the information to group the markers. This can cause a performance problem on the client, because all the information must be requested from the server for the clustering logic to take place.

During the reading of this article you will learn the basics to group the markers on the server and send them to the client in a minimized way. It can be mentioned that the Codegenio development team implemented this principle, obtaining the expected result.

Why server clustering?

Many developers prefer to use the Google Maps API as a map manager in their web applications. The Google Maps API has the advantage of having extensive documentation on its official site. This Library has support for clustering its markers (see the following example).

This is very good, but what would happen if we had to represent 100,000 markers, or what about 500,000 Humm, it is very likely that the browser does not respond because it has to make a request for 500,000 records to the server, create on client 500 000 markers and then group them to represent them on the map. This, as you may have noticed, is not entirely optimal, the ideal would be to send the client as little information as possible and represent all the data on the map.

In other words, it would basically be to perform the clustering process on the server and send the client a data structure that contains at least one position (lat, lng) and an integer representing the number of markers that exist in a given radius around the position. Our development team recently faced that problem and implemented this solution in the Reaquis application, in which its database grew and therefore had to represent many more locations on the map.

The solution

If we had to name an axis dependency for the solution of the problem, it would undoubtedly be the library that groups the markers on the server, in this case the focus will be on what Codegenio used to solve the problem in question. Well, the magic of clustering was done with the SuperCluster library, along with the GeoJson exchange format.

• GeoJson: Format for encoding a variety of geographic data structures.
• SuperCluster: A very fast JavaScript library for geospatial point clustering for browsers and Node.

Basically SuperCluster loads the data that includes the coordinates (lat, lng). Once loaded, the method that groups all this data for a given zoom level is invoked. All this input and output information for the SuperCluster library is done in GeoJSON format. Let’s look at the following code snippet that is nothing more than the logic of a SailsJs controller.

JavaScript

const Supercluster = require('supercluster');
const GeoJSON = require('geojson');
module.exports = {
 clusteredMarkers: async (req, res) => {
  try {
   let zoom = req.param('zoom');
   let data = await getData(req.allParams());
   let geoJsonData = GeoJSON.parse(data, {Point: ['lat', 'lng']});
   let index = new Supercluster({
       radius: 128,
       maxZoom: 18
    }).load(geoJsonData.features);
let clusteredData = index.getClusters([-180, -90, 180, 90], zoom);
res.send({
    success: true,
    total: clusteredData.length,
    records: clusteredData
   });
  } catch (err) {
   if (err) {
    return res.negotiate(err);
   }
  }
 }
};

Did you miss something? Do not worry, explain each part of the code.

• First, they are the SuperCluster dependencies and a library that converts any javascript object to GeoJSON. To install them npm install geojson and npm install supercluster.
• The zoom variable is obtained from the client and its value changes when the map changes, zooms in or out.
• The variable data is an array of objects that are get from the database. Here it will be assumed that the getData() method will go to the database and get the data from a collection. If SailsJs is used with MongoDb as a database it is recommended to make a native query, a reference of this can be seen here. The value of the data variable should be something like this:

JSON

[{
   "name": "mark 1",   
   "status": "STAUS-1",
   "street": "81 st"
    ...
   "lat": "55.91581954235805",
   "lng": "39.01556461897831",
},
  { .....
  }
]

• The geoJsonData variable is the result of parsing the data object, see that the second parameter specifies what data attributes the latitude and longitude contain for each element. The result would be an object with a collection of features as shown below:

JSON

{
 "type": "FeatureCollection":
 "features": [{
    "type": "Feature",
    "geometry": {
     "type": "Point",
     "coordinates": [125.6, 10.1]
      },
    "properties": {
      "name": "mark 1",
      "status": "STAUS-1",
      "street": "81 st",
       ...
      "lat": "55.91581954235805",
      "lng": "39.01556461897831"
      
      }     
     },....
 ]
}

• The index variable is an instance of SuperCluster and is loaded when invoking the load() method by passing the features of the GeoJson object.
• The clusteredData variable is the final result of the grouping, it is an array of features that will be sent to the client to create the markers on the map. Depending on the information of the feature a cluster or a marker will be displayed.

JSON

[
 {
 "type": "Feature",
 "id": 2120,
 "properties": {
   "cluster": true,
   "cluster_id": 2120,
   "point_count": 1800,
   "point_count_abbreviated": 1.8k
   },
 "geometry": {
   "type": "Point",
   "coordinates": [
      6.082657800000026,
      50.77498895307875
     ]
  }
},
{
 "type": "Feature",
 "properties": {  
  "name": "mark 1",
  "status": "STAUS-1",
  "street": "81 st"
  },
"geometry": {
    "type": "Point",
    "coordinates": [
       6.0771259,
       50.78158579999999
     ]
  }
},
...
]

So far what has been done is to obtain the data that will form the markers, convert it to the GeoJson format, load it into SuperCluster, cluster it and send it to the client.

Here the reader is only given an idea of how to group these markers on the server and send them to the client when an HTTP request arrives at the method.

How do we represent this in the client?

As previously mentioned, the Google Maps API is one of the most used by developers, so the focus will be based on this API. Surely you noticed that SuperCluster is a library developed by MapBox but do not worry that we can make it work with Google Maps, in addition the logic of the clusters has been made on the server allowing Google Maps not to know that we are using SuperCluster.

To represent the markers, the following must be done:

1. Create a map
2. Add an event that triggers when the visual limits of the map change
3. Make a request to our method on the server with the value of limits and zoom
4. Represent the markers on the map using the server information

JavaScript

const map = new google.maps.Map(document.getElementById('map'));
const markers = [];
map.addListener('bounds_changed', function () {
 const bounds = map.getBounds();
 const northEast = bounds.getNorthEast();
 const southWest = bounds.getSouthWest();
 const zoom = map.zoom;
 getDataFromServer(
    southWest.lng(), 
    southWest.lat(), 
    northEast.lng(), 
    northEast.lat(), 
    zoom
 ).then(function (res) {
  const geoJsonData = res.records;
  for (let data of geoJsonData) {
   if (data.properties.cluster) {
    markers.push(new google.maps.Marker({
      label: data.properties.point_count_abbreviated,
      icon: {
       path: google.maps.SymbolPath.CIRCLE        
      },
      map: map
     })
    );
} else {
    markers.push(new google.maps.Marker({
      label: data.properties.name,
      icon: {
       path: google.maps.SymbolPath.BACKWARD_OPEN_ARROW
      },
      map: map
     })
    );
   }
  }
 });
});

In this code these 4 aspects are covered, it is a basic example that the final result will be to show the grouped markers that have been grouped on the server.

If you are familiar with the Google Maps API it will not be difficult to understand the operation of the previous code. Note that it is assumed that the getDataFromServer () method is responsible for going to the server through an HTTP request to the clusteredMarkers method of our server and obtaining the information in GeoJson format to build the markers. This method was not implemented because it is not the objective of this article, but you can use some of the libraries used to make HTTP requests such as Request, axios or socket.io.

Conclusions

If you have reached this point, we can share the conclusion of this article, focusing on the fact that doing the process of grouping markers on the server will ensure that our application does not collapse when we zoom out of the map and a few thousand markers should be displayed. What is shown here is just an example that can be used for developers to apply this principle in modules that interact with maps that require displaying large amounts of information.

This article was originally posted at Codegenio’s blog by our Backend Engineer Dariel Noa.