Skip to content

SamGIS - Segment Anything applied to Geographic Information Systems (GIS)

This project, composed by a frontend (samgis-fe) and by a backend (samgis-be), is an attempt to perform machine learning instance segmentation on geo-spatial data even without the use of dedicated graphics cards.

I wrote the backend adapting SAM Exporter with the aim to use the machine learning segment anything project for the purpose to improve polygons recognition in GIS web applications.

These are the project resource links:

Note that the HuggingFace space demo has its static frontend files under the /static folder.

Ask me for a user via linkedin if you would like to test the authenticated demo or an explanation about the code.

SamGIS also replaces SurferDTM

SurferDTM was an attempt to find stratovolcanoes within digital terrain models (DEM/DTM). As a new feature SamGIS now can now use also input DEM images (like the first image in this gallery):

Note that the "terrarium" source provider data encode the image as RGB, so the backend processes it to obtain the elevation data (like the second image in this gallery) using this formula (from the map tile provider page):

(red * 256) + green + (blue / 256) - 32768

I also added a custom elaboration step to create an intermediate RGB image composed by channels:

  • RED - normalized DEM image
  • GREEN - normalized 2d composed image by
    • normalized DEM image
    • slope
    • curvature
  • BLUE - curvature

Segment Anything will finally do the inference on this rgb image.

Open this element detail to view the json request I use to create the DEM and RGB intermediate images.
json
{
   "bbox": {
         "ne": {"lat": 46.203706648934954, "lng": 9.401906739170105},
         "sw": {"lat": 46.12464369105346, "lng": 9.231103669101747}},
   "prompt": [
         {"id": 238, "type": "point", "data": {"lat": 46.158760519552146, "lng": 9.306795638666486}, "label": 1},
         {"id": 250, "type": "point", "data": {"lat": 46.152693546080975, "lng": 9.289288652508679}, "label": 0},
         {"id": 264, "type": "point", "data": {"lat": 46.16613515509541, "lng": 9.319840059725284}, "label": 0},
         {"id": 272, "type": "point", "data": {"lat": 46.161377438862836, "lng": 9.288087192674299}, "label": 0},
         {"id": 285, "type": "point", "data": {"lat": 46.174103408003454, "lng": 9.308340372739261}, "label": 0},
         {"id": 293, "type": "point", "data": {"lat": 46.17005996124035, "lng": 9.298042145587583}, "label": 0}
   ],
   "zoom": 13,
   "source_type": "nextzen.terrarium"
}

Project architecture

  1. The user interact with a WebMap (in my example it's a leaflet map that use OpenStreetMap map data).

  2. The user send a POST request to an API using

    1. the current map extent
    2. the current map zoom
    3. a prompt array of objects; objects could be
      • an "include" marker position (a couple of float - latitude and longitude)
      • an "exclusionary" marker position (a couple of float - latitude and longitude)
      • a "include" rectangle (two latitude and longitude positions - north-east and south-west)
  3. because of CORS constrain it's difficult to directly send requests to remote resources on different domains: to avoid this problems I prepared a simple Cloudflare proxy function that forward the user request to the remote API

  4. The API Gateway proxies the request to the serverless backend

  5. the serverless Lambda API elaborate the request:

    1. it parses the request and in particular translate the input latitude/longitude prompt into a x/y (pixel coordinates) one
    2. it downloads the geo-referenced tiles then merge and crop them
    3. it instantiates a MobileSam model instance if not already created using OnnxRuntime as runtime
    4. it prepares the image embedding and...
    5. ...it starts the image inference process
    6. it transforms the recognition masks into a single binary mask then it vectorizes the recognized binary mask:
      1. first it converts the mask into a GeoPandas GeoDataframe using rasterio.features.shapes and GeoDataFrame.from_features
      2. then it convert the new Geodataframe into a geojson using GeoDataFrame.to_json
  6. the Cloudflare proxy function parses the response and forward it to the frontend

  7. the frontend parse the geojson to load within the WebMap and shows some response info like the non-relativistic response duration time:

    datetime-diff

To avoid abuse the prompt input web map that I host on my personal site is under authentication (I use auth0.com).

AWS lambda, a serverless solution: some pros...

  • It can work also with docker images for complex or big projects (like this)
  • Almost easy, working out-of-the-box integration with other AWS services and third-parts authentication systems
  • Pay-as-you-go service

...and some cons

Like my website? Pay me a coffee
References are available upon request. I hereby authorize the use of my personal data in compliance with the Italian D. Lgs. 196/2003, art. 13 for the purpose of making me job offers.