The Decryption of SuperMap High-performance Distributed Map Rendering Technology

09 May,2019

Map service publishing is the basic function and requirement of GIS application system, as well as an important way of sharing data results.

At first, map service publishing employs the method of dynamic plotting. While with the data increasing, it takes longer time to dynamically plot map on the server. Therefore, GIS developers improve the efficiency of accessing map services by pre-generating map grid tiles.

However, as the increase of data amount and the improvement of map scale, the time of generating grid tiles for large-scale maps takes longer. Moreover, if the map data and map style changed, the new grid tiles need to be generated. It will be moreinconvenient to update.

In order to better solve the user's needs for real-time update, instant release and efficient browsing of spatial data, SuperMap brings over big data technology and integrates advanced technologies of distributed storage, vector pyramid, distributed rendering and automatic caching to create a high-performance distributed map rendering technology solution and to achieve HBase-based ultra-large-scale data map-free publishing service. With this technology, it only takes hours to complete the data storage, the data can be published without pre-cache action, and the quickly map dynamic response can be realized.

Key Technologies for High Performance Distributed Rendering

Distributed storage technology
Distributed storage technology can effectively solve the limitations of traditional relational databases in ultra-large-scale data management. First, the relational database is difficult to cope with the query and analysis of huge records. As the increase ofusers, hard disk read-write will become a challenge. In particular, relational databases are weak in terms of scalability and availability, and the cost is relatively high. Based on the above analysis, relational databases have been difficult to meet the storage requirements of large amounts of data.

The distributed technology can realize the horizontal expansion through the cluster's distributed processing method to conduct operations to big data, such as horizontal split (to averagely distribute data to multiple database nodes), so that the amount of data will be smaller and the storage management performance will be improved. In addition, the distributed capabilities of mainstream distributed databases are transparent to users and can seamlessly conform to the user's SQL operating habits, which becomes more convenient for users to use and manage.

HBase distributed database is based on HDFS and is an open source, distributed, versioned non-relational database. The core storage model is built based on Google's BigTable, which targets on low-cost, scalable hardware devices to host billion-row rows and millions of columns. It is a modular design that supports horizontal scaling and automatic table sharding, as well as automatic failover between servers in different regions.

The HBase cluster can be registered into SuperMap iServer. By using the “Copying Data” function of SuperMap iServer, the data of the local UDB, GDB file, shape file and the data registered to SuperMap iServer can be easily completed and imported into the HBase database. It can realize the writing performance of 110,000 polygon objects per second and provide a solid foundation for massive data migration to distributed storage.

Distributed rendering technology
High-performance distributed rendering technology supports the direct release of massive data services without slices. SuperMap has developed distributed rendering technology, which realizes the decomposition of the requested vector tile rendering task on the SuperMap iServer server, and executes it by multiple processes to make full and more efficient use of computing resources. At the same time, the technology can further configure the SuperMap iServer cluster and send the block rendering task to the cluster sub-nodes to further improve the parallelism of the calculation. With the combination of multi-process and cluster, the rendering performance can be greatly improved and the second-response efficiency of ultra-large-scale data can be realized.

Vector pyramid technology
The above technologies solve the problem of efficient rendering of large-scale data, while it is difficult to achieve a second-level response when the billion-level data is displayed on a small scale or even a full-frame display. To solve the problem, SuperMap has developed vector pyramid technology, which is similar to the image pyramid. The technology segments vector data from multi-scale and obtains a series of data sets whose data precision is gradually reduced in a pyramid shape. The bottom of the pyramid is the original level of vector data, while the top is a low-precision approximation of vector data. The technology solves the problem of the massive vector data second-level displaying on small scale and reduces the complexity of data drawing in the case of ensuring the displaying effect, thereby greatly improving the plotting performance.

Automatic caching
The automatic caching technology can further enhance the client map accessing efficiency. By automatically invoking the data browsed by the user, the server no need to render the data when browsing the area again, which directly displays the cached results.