Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add support for STAC API Transactions #1054

Merged
merged 2 commits into from
Dec 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/stac.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,14 @@ STAC support will render links as follows:
* links that are enclosures will be encoded as STAC assets (in ``assets``)
* all other links remain as record links (in ``links``)

Transactions
^^^^^^^^^^^^

STAC Transactions are supported as per the following STAC API specifications:

* `STAC API - Transaction Extension Specification`_.
* `STAC API - Collection Transaction Extension`_.

Request Examples
----------------

Expand Down Expand Up @@ -106,3 +114,5 @@ Request Examples
http://localhost:8000/stac/collections/metadata:main/items/{itemId}

.. _`SpatioTemporal Asset Catalog API version v1.0.0`: https://github.com/radiantearth/stac-api-spec
.. _`STAC API - Transaction Extension Specification`: https://github.com/stac-api-extensions/transaction
.. _`STAC API - Collection Transaction Extension`: https://github.com/stac-api-extensions/collection-transaction
41 changes: 41 additions & 0 deletions docs/transactions.rst
Original file line number Diff line number Diff line change
Expand Up @@ -114,5 +114,46 @@ Harvesting

Harvesting is not yet supported via OGC API - Records.

Transactions using STAC API
===========================

pycsw's STAC API support provides transactional capabilities via the `STAC API - Transaction Extension Specification`_ and `STAC API - Collection Transaction Extension`_ specifications,
which follows RESTful patterns for insert/update/delete of resources.

Supported Resource Types
------------------------

STAC Collections, Items and Item Collections are supported via OGC API - Records transactional workflow. Note that the HTTP ``Content-Type``
header MUST be set to (i.e. ``application/json``).

Transaction operations
----------------------

The below examples demonstrate transactional workflow using pycsw's OGC API - Records endpoint:

.. code-block:: bash

# insert STAC Item
curl -v -H "Content-Type: application/json" -XPOST http://localhost:8000/stac/collections/metadata:main/items -d @fooitem.json

# update STAC Item
curl -v -H "Content-Type: application/json" -XPUT http://localhost:8000/stac/collections/metadata:main/items/fooitem -d @fooitem.json

# delete STAC Item
curl -v -XDELETE http://localhost:8000/stac/collections/metadata:main/items/fooitem

# insert STAC Item Collection
curl -v -H "Content-Type: application/json" -XPOST http://localhost:8000/stac/collections/metadata:main/items -d @fooitemcollection.json

# insert STAC Collection
curl -v -H "Content-Type: application/json" -XPOST http://localhost:8000/stac/collections -d @foocollection.json

# update STAC Collection
curl -v -H "Content-Type: application/json" -XPUT http://localhost:8000/stac/collections/foocollection -d @foocollection.json

# delete STAC Collection
curl -v -XDELETE http://localhost:8000/stac/collections/foocollection

.. _`OGC API - Features - Part 4: Create, Replace, Update and Delete`: https://docs.ogc.org/DRAFTS/20-002.html
.. _`STAC API - Transaction Extension Specification`: https://github.com/stac-api-extensions/transaction
.. _`STAC API - Collection Transaction Extension`: https://github.com/stac-api-extensions/collection-transaction
1 change: 1 addition & 0 deletions pycsw/core/repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,7 @@ def __init__(self, database, context, app_root=None, table='records', repo_filte
self.query_mappings = {
'identifier': self.dataset.identifier,
'type': self.dataset.type,
'typename': self.dataset.typename,
'parentidentifier': self.dataset.parentidentifier,
'collections': self.dataset.parentidentifier,
'updated': self.dataset.insert_date,
Expand Down
3 changes: 2 additions & 1 deletion pycsw/ogc/api/records.py
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,8 @@ def get_response(self, status, headers, data, template=None):
if headers.get('Content-Type') == 'text/html' and template is not None:
content = render_j2_template(self.config, template, data)
else:
content = to_json(data)
pretty_print = str2bool(self.config['server'].get('pretty_print', False))
content = to_json(data, pretty_print)

headers['Content-Length'] = len(content)

Expand Down
42 changes: 37 additions & 5 deletions pycsw/stac/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,9 @@
'https://api.stacspec.org/v1.0.0/item-search#filter',
'https://api.stacspec.org/v1.0.0/item-search#free-text',
'https://api.stacspec.org/v1.0.0-rc.1/collection-search',
'https://api.stacspec.org/v1.0.0-rc.1/collection-search#free-text'
'https://api.stacspec.org/v1.0.0-rc.1/collection-search#free-text',
'https://api.stacspec.org/v1.0.0/collections/extensions/transaction',
'https://api.stacspec.org/v1.0.0/ogcapi-features/extensions/transaction'
]


Expand Down Expand Up @@ -321,11 +323,15 @@ def collection(self, headers_, args, collection='metadata:main'):
if collection == 'metadata:main':
collection_info = self.get_collection_info()
else:
virtual_collection = self.repository.query_ids([collection])[0]
collection_info = self.get_collection_info(
virtual_collection.identifier,
dict(title=virtual_collection.title,
try:
virtual_collection = self.repository.query_ids([collection])[0]
collection_info = self.get_collection_info(
virtual_collection.identifier,
dict(title=virtual_collection.title,
description=virtual_collection.abstract))
except IndexError:
return self.get_exception(
404, headers_, 'InvalidParameterValue', 'STAC collection not found')

response = collection_info
url_base = f"{self.config['server']['url']}/collections/{collection}"
Expand Down Expand Up @@ -456,6 +462,11 @@ def item(self, headers_, args, collection, item):
return self.get_exception(400, headers_, 'InvalidParameterValue', msg)

response = json.loads(response)

if 'id' not in response:
return self.get_exception(
404, headers_, 'InvalidParameterValue', 'item not found')

response = links2stacassets(collection, response)

return self.get_response(status, headers_, response)
Expand Down Expand Up @@ -497,6 +508,27 @@ def get_collection_info(self, collection_name: str = 'metadata:main',
}]
}

def manage_collection_item(self, headers_, action='create', item=None, data=None, collection=None):
if action == 'create' and 'features' in data:
LOGGER.debug('STAC Collection detected')

for feature in data['features']:
data2 = feature
if collection is not None:
data2['collection'] = collection

headers, status, content = super().manage_collection_item(
headers_=headers_, action='create', data=data2)

return self.get_response(201, headers_, {})

else: # default/super
if collection is not None:
data['collection'] = collection

return super().manage_collection_item(
headers_=headers_, action=action, item=item, data=data)


def links2stacassets(collection, record):
LOGGER.debug('Transforming enclosure links to STAC assets')
Expand Down
43 changes: 32 additions & 11 deletions pycsw/wsgi_flask.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,6 @@
with open(os.getenv('PYCSW_CONFIG'), encoding='utf8') as fh:
APP.config['PYCSW_CONFIG'] = yaml_load(fh)

pretty_print = APP.config['PYCSW_CONFIG']['server'].get('pretty_print', True)
APP.config['JSONIFY_PRETTYPRINT_REGULAR'] = pretty_print

BLUEPRINT = Blueprint('pycsw', __name__, static_folder=STATIC,
static_url_path='/static')

Expand Down Expand Up @@ -137,8 +134,8 @@ def conformance():
return get_response(api_.conformance(dict(request.headers), request.args))


@BLUEPRINT.route('/collections')
@BLUEPRINT.route('/stac/collections')
@BLUEPRINT.route('/collections', methods=['GET', 'POST'])
@BLUEPRINT.route('/stac/collections', methods=['GET', 'POST'])
def collections():
"""
OGC API collections endpoint
Expand All @@ -147,13 +144,19 @@ def collections():
"""

if get_api_type(request.url_rule.rule) == 'stac-api':
return get_response(stacapi.collections(dict(request.headers), request.args)) # noqa
if request.method == 'POST':
data = request.get_json(silent=True)
return get_response(stacapi.manage_collection_item(dict(request.headers),
'create', data=data))
else:
return get_response(stacapi.collections(dict(request.headers),
request.args))
else:
return get_response(api_.collections(dict(request.headers), request.args))


@BLUEPRINT.route('/collections/<collection>')
@BLUEPRINT.route('/stac/collections/<collection>')
@BLUEPRINT.route('/collections/<collection>', methods=['GET', 'PUT', 'DELETE'])
@BLUEPRINT.route('/stac/collections/<collection>', methods=['GET', 'PUT', 'DELETE'])
def collection(collection='metadata:main'):
"""
OGC API collection endpoint
Expand All @@ -164,8 +167,18 @@ def collection(collection='metadata:main'):
"""

if get_api_type(request.url_rule.rule) == 'stac-api':
return get_response(stacapi.collection(dict(request.headers),
request.args, collection))
if request.method == 'PUT':
return get_response(
stacapi.manage_collection_item(
dict(request.headers), 'update', collection,
data=request.get_json(silent=True)))
elif request.method == 'DELETE':
return get_response(
stacapi.manage_collection_item(dict(request.headers),
'delete', collection))
else:
return get_response(stacapi.collection(dict(request.headers),
request.args, collection))
else:
return get_response(api_.collection(dict(request.headers),
request.args, collection))
Expand Down Expand Up @@ -203,14 +216,22 @@ def items(collection='metadata:main'):
:returns: HTTP response
"""

if request.method == 'POST' and request.content_type not in [None, 'application/json']: # noqa
if all([get_api_type(request.url_rule.rule) == 'ogcapi-records',
request.method == 'POST',
request.content_type not in [None, 'application/json']]):

data = None
if request.content_type == 'application/geo+json': # JSON grammar
data = request.get_json(silent=True)
elif 'xml' in request.content_type: # XML grammar
data = request.data

return get_response(api_.manage_collection_item(dict(request.headers),
'create', data=data))
elif request.method == 'POST' and get_api_type(request.url_rule.rule) == 'stac-api':
data = request.get_json(silent=True)
return get_response(stacapi.manage_collection_item(dict(request.headers),
'create', data=data, collection=collection))
else:
if get_api_type(request.url_rule.rule) == 'stac-api':
return get_response(stacapi.items(dict(request.headers),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ def test_queryables(config):
assert content['$id'] == 'http://localhost/pycsw/oarec/collections/metadata:main/queryables' # noqa
assert content['$schema'] == 'http://json-schema.org/draft/2019-09/schema'

assert len(content['properties']) == 13
assert len(content['properties']) == 14

assert 'geometry' in content['properties']
assert content['properties']['geometry']['$ref'] == 'https://geojson.org/schema/Polygon.json' # noqa
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ def test_queryables(config_virtual_collections):
assert content['$id'] == 'http://localhost/pycsw/oarec/collections/metadata:main/queryables' # noqa
assert content['$schema'] == 'http://json-schema.org/draft/2019-09/schema'

assert len(content['properties']) == 13
assert len(content['properties']) == 14

assert 'geometry' in content['properties']
assert content['properties']['geometry']['$ref'] == 'https://geojson.org/schema/Polygon.json' # noqa
Expand Down
Loading