Joining data to a layer source

[donnyv]

I have an issue with a project I'm working on. I have data stored in Postgis and I'm serving it out as MVT tiles using Marten. But I would like to be able to dynamically change the attribute data that gets served out with the tiles. Now I could do this on the postgres side by creating a function inside the code that serves the MVT tiles. But I would like to keep the attribute data stored in a different database. It would be handy if I could move this client side. By providing the layer source a join data link. This join link could be an array of object data that has an id with values that match with the MVT tile attribute id. When it renders it could look up the data in the array and use it to do attribute rendering and spatial operations. This join data could be created client side or fetched from somewhere else. It also keeps the MVT Tile size small since it doesn't need to carry the attribute data anymore. It could also be cached for the entire layer when the site first loads.

[HarelM]

Can you give a concrete example?
this might be done using addProtocol, but it won't be fast...

[donnyv]

Something like this...

var extendedData = [
	{
		'Id': 10001,
		'BuildingSqft': "1800",
		'ClassColor': "blue"
	},
	{
		'Id': 10002,
		'BuildingSqft': "900",
		'ClassColor': "green"
	}
];

map.addSource('parcel-source', {
    'type': 'geojson',
    'data': 'https://www.site.com/data/parcels.geojson',
    'data-ext': {
        'link-id': 'Id',
        'data': extendedData
     }
});

map.addLayer({
	'id': 'parcels',
	'type': 'fill',
	'source': 'parcel-source',
	'paint': {
		'fill-color': ['get', 'data-ext.ClassColor']
	}
});

[donnyv]

This would help with tile size in mvt caches. Especially if the user is using a CDN. It would bring down cost and they could use 1 layer cached but have different datasets linked to it.

[HarelM]

At this point, your best option is addProtocol. Your other option is to use importScriptInWorkers, but that would require to understand the internals, which would be very time consuming.

[donnyv]

Does addProtocol give you access to the render pipeline? I'm assuming I would need to do the lookup before the render and alter it based on the look up.

[HarelM]

No, it doesn't.

[cutephoton]

I'm encountering something similar. My use-case has features (33k) that don't change, but attributes that change in the time domain (and are updated regularly in batches). I started by creating new tiles for each time slice but quickly discovered the problem with this approach. A single slice is ~16mb (mbtile) representing an hour which sounds reasonable to me. However, the entire dataset encoded into separate self-contained slices is 2gb. I think it would work well enough for a client with a good network connection, but I am not a fan.

I wonder if, in some applications, the vector tile would be better represented as independent feature and attribute tiles (or a hybrid). This would be useful for situations where attributes are expected to change, are dynamic, or temporal, but the features are not. I took a look around to see how feasible this would be. I decided it would be pretty hard for somebody unfamiliar with the codebase. It was interesting to explore.

[donnyv]

I feel like a feature that separates the vector shape data from the attribute's could bring a lot of efficiency to a layer. Especially if its across a lot of layers.

[andrewharvey]

I feel like a feature that separates the vector shape data from the attribute's could bring a lot of efficiency to a layer. Especially if its across a lot of layers.

There is a proposal at maplibre/maplibre-tile-spec#312 to allow that with MapLibre Tiles, but it would still need support here too.