Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coerce tags to lower-case strs #244

Merged
merged 14 commits into from
Apr 28, 2017
12 changes: 4 additions & 8 deletions datafs/core/data_archive.py
Original file line number Diff line number Diff line change
Expand Up @@ -798,19 +798,15 @@ def add_tags(self, *tags):
'''
Set tags for a given archive
'''

for tag in tags:
assert isinstance(tag, string_types), 'tags must be strings'

self.api.manager.add_tags(self.archive_name, tags)
normed_tags = self.api.manager._normalize_tags(tags)
self.api.manager.add_tags(self.archive_name, normed_tags)

def delete_tags(self, *tags):
'''

Deletes tags for a given archive

'''
for tag in tags:
assert isinstance(tag, string_types), 'tags must be strings'
normed_tags = self.api.manager._normalize_tags(tags)

self.api.manager.delete_tags(self.archive_name, tags)
self.api.manager.delete_tags(self.archive_name, normed_tags)
19 changes: 19 additions & 0 deletions datafs/managers/manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,9 @@ def _create_archive_metadata(
if tags is None:
tags = []

else:
tags = self._normalize_tags(tags)

check_requirements(
to_populate=user_config,
prompts=self.required_user_config,
Expand Down Expand Up @@ -485,6 +488,22 @@ def delete_tags(self, archive_name, tags):

self._set_tags(archive_name, updated_tag_list)

def _normalize_tags(self, tags):
'''
Coerces tags to lowercase strings

Parameters
----------
tags: list or tuple of strings

'''

lowered_str_tags = []
for tag in tags:
lowered_str_tags.append(str(tag).lower())

return lowered_str_tags

def _get_archive_spec(self, archive_name):
res = self._get_archive_listing(archive_name)

Expand Down
2 changes: 1 addition & 1 deletion docs/api/cli/cli.tagging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ View the source for the code samples on this page in :ref:`snippets-cli-tagging`
Tagging on archive creation
---------------------------

When creating archives, you can specify tags using the ``--tag`` argument. You can specify as many as you would like:
When creating archives, you can specify tags using the ``--tag`` argument. Tags must be strings and will be coerced to lowercase as a standard. You can specify as many as you would like:

.. include:: ../../../examples/snippets/cli_tagging.py
:start-after: .. EXAMPLE-BLOCK-1-START
Expand Down
2 changes: 1 addition & 1 deletion docs/api/python/pythonapi.tagging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ View the source for the code samples on this page in :ref:`snippets-pythonapi-ta
Tagging on archive creation
---------------------------

When creating archives, you can specify tags using the ``tags`` argument. You can specify as many as you would like as elements in a list:
When creating archives, you can specify tags using the ``tags`` argument. Tags must be strings and will be coerced to lowercase as a standard. You can specify as many as you would like as elements in a list:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to update the CLI version of this document as well.


.. include:: ../../../examples/snippets/pythonapi_tagging.py
:start-after: .. EXAMPLE-BLOCK-1-START
Expand Down
2 changes: 2 additions & 0 deletions docs/whatsnew.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ What's New

These are new features and improvements of note in each release.

.. include:: whatsnew/v0.7.1.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've seen other packages only include the whatsnew of published versions, so this would be included in the TOC once 0.7.1 is released.


.. include:: whatsnew/v0.7.0.txt

.. include:: whatsnew/v0.6.9.txt
Expand Down
49 changes: 40 additions & 9 deletions docs/whatsnew/v0.7.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,14 @@ v0.7.1 (Latest Build)
New features
~~~~~~~~~~~~

- Archive names are normalized in DataAPI methods. See :ref:`normalizing archive names <normalizing_archive_names>` (:issue:`220` & :issue:`235`).
- Archive names are normalized in DataAPI methods. See `Normalizing archive names`_ (:issue:`220` & :issue:`235`).

- Tags are now normalized to lowercase strings. See `Normalize tags`_ (:issue:`243`).



.. _Normalizing archive names:

.. _normalizing_archive_name
Normalizing archive names
^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down Expand Up @@ -42,14 +47,38 @@ the archive_name or archive created in the above examples:

Search patterns do not accept authority names:

.. code-block:: python

>>> api.search(prefix='authority://my') # no results


.. _Normalize tags:

Normalize Tags
^^^^^^^^^^^^^^

On :py:class:`~datafs.DataAPI` method :py:meth:`~datafs.DataAPI.create`, and
:py:class:`~datafs.DataArchive` method :py:meth:`~datafs.DataArchive.add_tags` and
:py:meth:`~datafs.DataArchive.get_tags`.

.. code-block:: python

>>> arch1 = api.create('my_archive', tags=['TAG1', 'tag2', 42])
>>> arch1.get_tags()
['tag1', 'tag2', '42']
>>>
>>> arch1.add_tags('tAg4', 21)
>>> arch1.get_tags()
['tag1', 'tag2', '42', 'tag4', '21']


Archive name checking
.....................
^^^^^^^^^^^^^^^^^^^^^

The normalization process catches illegal archive names:

.. code-block:: python

>>> api.create('!!!\\\\~')
Traceback (most recent call last):
...
Expand All @@ -73,6 +102,8 @@ Backwards incompatible API changes
under any circumstances (:issue:`220` and :issue:`235`).




Performance Improvements
~~~~~~~~~~~~~~~~~~~~~~~~

Expand All @@ -82,17 +113,17 @@ Bug Fixes

- Implement missing ``default_version`` handling in :py:meth:`~datafs.DataAPI.get_archive` and :py:meth:`~datafs.DataAPI.batch_get_archive` (:issue:`240`)
- Messages are now coerced to strings, and :py:meth:`~datafs.core.data_archive.log` and the CLI ``log`` command no longer fail when used on archives with non-string messages (:issue:`232`)
- ``examples/ondisk.py`` updated to reflect [xarray 0.9.5](https://pypi.python.org/pypi/xarray/0.9.5) display function changes (:issue:`249`)
- ``examples/ondisk.py`` updated to reflect `xarray 0.9.5 <https://pypi.python.org/pypi/xarray/0.9.5) display function changes>`_ (:issue:`249`)
- Configuration now creates the datafs app directory if it did not previously exist (:issue:`265`)


Under the hood
~~~~~~~~~~~~~~

- Use ``:issue:`` and ``:pull:`` directives to reference a github issue or pull request (:issue:`209`)
- The sphinx build is now tested on travis. Run the tests locally with the command ``sphinx-build -W -b html -d docs/_build/doctrees docs/. docs/_build/html`` (:issue:`211`)
- The docs structure has been reorganized
- Conda dependencies pinned in ``requirements_conda.txt``, and the channel ``conda-forge`` was added to the travis conda environment so we have access to the latest conda builds. (:issue:`247`)
- Running the ``configure`` command not creates an empty 'default' profile if no configuration file exists
- Use ``:issue:`` and ``:pull:`` directives to reference a github issue or pull request (:issue:`209`)
- The sphinx build is now tested on travis. Run the tests locally with the command ``sphinx-build -W -b html -d docs/_build/doctrees docs/. docs/_build/html`` (:issue:`211`)
- The docs structure has been reorganized
- Conda dependencies pinned in ``requirements_conda.txt``, and the channel ``conda-forge`` was added to the travis conda environment so we have access to the latest conda builds. (:issue:`247`)
- Running the ``configure`` command not creates an empty 'default' profile if no configuration file exists

See the issue tracker on GitHub for a complete list.
15 changes: 13 additions & 2 deletions examples/snippets/pythonapi_tagging.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,14 +71,25 @@

>>> archive1 = api.create(
... 'archive1',
... tags=["foo", "bar"],
... tags=["foo", "Bar"],
... metadata={'description': 'tag test 1 has bar'})
...
>>> archive1.get_tags()
[u'foo', u'bar']
>>>
>>> archive2 = api.create(
... 'archive2',
... tags=["foo", "baz"],
... tags=["foo", "Baz"],
... metadata={'description': 'tag test 1 has baz'})
...
>>> archive2.get_tags()
[u'foo', u'baz']
>>>
>>> archive2.add_tags(42)
>>> archive2.get_tags()
[u'foo', u'baz', u'42']



.. EXAMPLE-BLOCK-1-END

Expand Down
13 changes: 8 additions & 5 deletions tests/test_archive_search.py
Original file line number Diff line number Diff line change
Expand Up @@ -168,27 +168,30 @@ def test_fn_search(api_with_diverse_archives):


def test_tagging_update_and_get(api_with_spec):
'''
Tests updated to reflect :issue:`243`
'''

api_with_spec.create(
'tagged_archive', metadata=dict(description='archive description'))
'tagged_archive', metadata=dict(description='archive description'),
tags=['Tag1', 'TAG2'])

arch = api_with_spec.get_archive('tagged_archive')
arch.add_tags('tag1', 'tag2')

res = api_with_spec.search('tag1', 'tag2')

assert 'tagged_archive' in list(res)

arch.add_tags('tag3', 'tag1')
arch.add_tags('tag3', 'tag1', 42)

result = arch.get_tags()
assert 'tag1' and 'tag2' and 'tag3' in result
assert len(result) == 3
assert len(result) == 4

arch.delete_tags('tag1')
result_delete = arch.get_tags()
assert 'tag1' not in result_delete
assert len(result_delete) == 2
assert len(result_delete) == 3


def test_begins_with_filter(api_with_diverse_archives):
Expand Down