[Part 13] Tackling Terabyte-Level Data! iDesktopX Online Big Data Analysis

Hello! I'm ippuku_time, a GIS implementation support consultant.

This is the 13th installment of the "5-Minute Guide to SuperMap iDesktopX Products and Features" series. Last time, we explored facility network analysis used for water and power grids. This time, we’re finally stepping into the world of big data.

How does iDesktopX analyze terabytes of spatial data—such as nationwide smartphone location information or full vehicle trajectory logs—that a single PC could never process?
Let’s uncover the secret.

1. Big Data Online Analysis Mechanism

When you hear “big data analysis,” you may imagine complex programming, specialized tools, or a heavy computational environment. But iDesktopX takes a different approach.

iDesktopX acts as a comfortable terminal where you issue analysis commands and view results.

The actual heavy processing happens behind the scenes through:

• the powerful GIS server SuperMap iServer, and

• the distributed computing platform Apache Spark.

This architecture lets users maintain a familiar desktop GIS workflow while tapping into the power of server clusters to handle massive-scale spatial data.

2. Preparing for Analysis: Connecting to iServer

Before starting an analysis, connect iDesktopX to an iServer instance with big data analysis enabled.

The large datasets must be stored in distributed storage systems—such as HDFS or PostgreSQL—and then registered in iServer for processing.

3. Representative Big Data Analysis Functions

iDesktopX’s big data tools behave much like typical spatial analysis tools, but are optimized to handle terabyte-level datasets.

(1) Density Analysis

Instantly visualizes high-density areas (hotspots) from tens or hundreds of millions of points.

Use case:

Visualize nationwide location-enabled social media posts to show where crowds gathered during an event (e.g., fireworks festival).

(2) Aggregation Analysis

Aggregates massive point or line datasets into grids (e.g., 1 km mesh) or polygon units (cities, districts, etc.).

Use case:

Aggregate billions of vehicle GPS points into 1 km meshes to generate a national traffic volume map.

(3) OD Line Construction

Generates origin–destination (OD) lines to visualize flows of people or goods.

Use case:

Analyze tens of millions of transportation IC card records to identify the most common commuter routes in the Tokyo metropolitan area.

(4) Trajectory Reconstruction

Reconstructs chronological movement routes from unordered GPS point logs.

Use case:

Rebuild daily movement paths of thousands of delivery trucks from randomly transmitted GPS points to eva1uate delivery efficiency.

Summary

Today, we introduced the big data online analysis capabilities made possible by SuperMap iDesktopX’s server-based processing.

iDesktopX allows anyone—not just specialists—to analyze massive datasets while maintaining the ease and usability of desktop GIS.

This unlocks large-scale spatial data analysis for fields such as smart cities, transportation, and MaaS (Mobility as a Service).

Next time, in Part 14, we’ll explore map tiles—the key to fast web map display.

We’ll explain how map tiles work and how to create and manage them. Stay tuned!

For source and sample code: https://supermap-japan.blogspot.com/2025/09/13idesktopx.html