Clean up some tests for programmatic descriptions feature #407

ttannis · 2020-03-17T00:39:48Z

Summary of Changes

Some cleanup on top of #381

Python unit tests were updated to explicitly test the behavior of helper functions added in metadata_utils
Placed hardcoded text from TableDetail into a constants file. We are separating out hardcoded user-readable text for out components because it will make them easier to update in the future -- especially if we eventually allow for constants for be overwritten.
Updated the test for the EditableSection title to verify that convertText() is called and the result of that method is what gets rendered.

Tests

Included above.

Documentation

N/A. Features have not been modified.

CheckList

Make sure you have checked all steps below to ensure a timely review.

PR title addresses the issue accurately and concisely. Example: "Updates the version of Flask to v1.0.2"
- In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.
PR includes a summary of changes, including screenshots of any UI changes.
PR adds unit tests, updates existing unit tests, OR documents why no test additions or modifications are needed.
In case of new functionality, my PR adds documentation that describes how to use it.
- All the public python functions and the classes in the PR contain docstrings that explain what it does
PR passes all tests documented in the developer guide

ttannis · 2020-03-17T00:44:54Z

@samshuster as promised, check out this PR for some minor cleanup I added on top of your PR in order to make tests a little more robust.

samshuster · 2020-03-17T15:33:14Z

thanks @ttannis this was useful!

* Allow users to create JIRA tickets associated with a table (#389) * add jira connection * update location of report data issue * add some reducers * add to global state * fix saga/redux flow for the jira issues view * use redux to handle reporting a data issue : * fix build * add jira sdk to requirements.txt, run fix lint errors for js * add comment * add data quality icon * move constants into a separate file * add tests for reducer and api * add some more tests * add more tests, do some refactoring * still working on tests * fix unit tests that don't quite work * fix lint errors * add some python tests * add more tests for jira client * remove unused imports * remove unused import * fix lint errors * for some reason i have to add types? * re-fix tests * update comments * address some comments * respond to comments * respond to comments * remove jira project id requirement: * add base class for issue tracking * add resource for issue endpoint * interim commit while refactoring into more generic * do some refactor, add a generic base client * refactor name from jira -> issue * fix spec per changes in the api * respond to some pr comments, add some error handling if no jira component is enabled * update per pr comments * remove utils for trucation, use ellipsis * add custom exception class, make sure it returns config error on api call * fix tests, refactor * fix lint * hopefully fix build * css fix * try removing jira from requirements * update tests, add total count of remaining results to api call * add comment, see if this fixes the build * update tests * update build steps * try making the extras a string like in examples * try to follow example exactly * try without quotes * update tests, make method to generate url * try string again * try using jira as a string * remove references to extras and just install the package * mock urllib * update tests * add tests for rending extra issues * update text on issues view * fix some lint issues * respond to pr comments * update per pr comments * respond to pr comments * fix some lint errors * respond to comments, some lint fixes * fix more lint * respond to pr comments * remove todo * add definition of api calls in fe * hide issue integration by default * fix tests, increase test coverage a bit * update some text * update per some product feedback * update test * Apply suggestions from code review Co-Authored-By: Tamika Tannis <[email protected]> * update per pr comments Co-authored-by: Tamika Tannis <[email protected]> * Add documenation for JIRA feature (#395) * Add documenation for JIRA feature * update link * update docs a little bit * add some more clarification * remove whitespace * Search Filtering Support (#384) * Initial Filter React/Redux Modifications (#357) * Update configs and SearchFilter UI * Remove search syntax logic in SearchBar; Add RegEx validatation for SearchBar * Add filter state and ducks logic * Update filter UI; Code cleanup * Some python lint fixes * Code cleanup * Update request utils * Filter UX Improvements (#361) * SearchFilter UX improvements * Filter url updates; Modify no search results message * Support loading filters from pasted url * InputFilter apply changes on 'enter' * Simplify reducer logic + lint cleanup * Some cleanup based on comments * Fix build * Mypy fix * Mypy fix * Filter UI Improvements: Part 1 (#370) * Change 'Type' to 'Source' * Prevent SearchPanel width increase when InputFilter is present * Re-lower coverage while in development * Managed input autocomplete (#375) * Disabled autocomplete for some email input fields * Removed the hard coded font size for autocomplete * Improve user search experience (#376) * Fix a merge issue * Allow no search term; Persist filters on SearchPage * Lint fix * Add a TODO * Add support for configurable filter section help text (#377) * Add support for configurable filter section help text * Support on CheckboxFilterSection as well * Add default filterCategories for tables * Add default helpText * Update search flow (some more) (#383) * Add a Breadcrumb on HomePage that makes filters discoverable * Switch to using explicit clearing of search term; Revert to requiring term for SearchBar submit * Allow for html in InforButton text * WIP: Comment out some tests * Fix an edge case * Cleanup python logic; Add comments; Add tests; * Fix indentation error; Cleanup python tests; * Cleanup HomePage, Breadcrumb, InfoButton; Update tests * Connect SearchBar to router for executing special SearchPage logic * Cleanup SearchBar; Add tests; * Cleanup SearchPage; Add tests * Cleanup SearchFiilter components; Add tests; Increase jest threshold; * Fix build; Update FilterSection types * Cleanup config; Add tests; Update jest threshold; * Add another config test * Fix SearchFilter; Add filter action types; Code cleanup; Add tests; * Cleanup search reducer; Update/add tests; * Cleanup ducks + utils; Add tests; * Update application config documentation * Update endpoint to match search library change * Remove unused lines * Update input styles * rename filterconfig 'value' reference to 'categoryId' * Add new filter logic to support TagInfo case * Lint fix * Restructure filter components to leverage mapStateToProps; Update FilterType enums * Lint fix * Fix bad merge * Apply suggestions from code review Co-authored-by: Daniel <[email protected]> * Adds some responsiveness to table detail layouts (#391) * Improve granularity of logging search actions (#396) * WIP: One approach * Second approach * Fix a few errors; Add tests * Fix test * Code cleanup * Change value for when user selects inline result & searchAll is needed * Update SearchType enum * Use snake_case for consistency in backend until we have some auto-convert * Filter Fixes (#399) * Reset pageIndex to 0 on filter update * Allow CheckBoxItem to handle checked = undefined * Create TableHeaderBullets component (#397) * Create TableHeaderBullets component * Fix mocks * Update search_table endpoint (#400) * Update search_table endpoint * Update test docstring * Add reporter for JIRA Issue Creation (#401) * WIP test commit * Update reporter object * Testing user id * Another test * Update fix * Update jira logic. Move default configs * Add issue link; Add reported by in description * Update logic to transform filters then call _search_table (#402) * Update logic to transform filters then call _search_table * Update tests * Upgrade redux & react-redux (#373) * Update search results to include badges and display them (#398) * update search to include badges * fix test * slight change * remove mapping for badges * fix lint * remove badge as a search term since its not in the filters in ui * update test, remove unneeded import * Caste input to lowercase like we do for SearchBar (#404) * Issue 147/multi doc pathways (#381) * This adds "programmatic descriptions" into the frontend. Known issues: - is_editable property not respecected Added NonEditable Section Now no longer need to sort in index page. Uses the display title for title now. Added currently untested code in the api for programmatic display programmatic descriptions no longer needs to be returned by metadata service for backwards compatibility Tests added for the programmatic display component Caught a bug where display_title was not being set if configuration wasn't set. change source_id to source Make code more robust Rebasing with the upstream changes that have been made. adding documentation removing the comment in the config.py class Fixing upstream merge conflicts. Update docs/flask_config.md Co-Authored-By: jornh <[email protected]> Removing non editable section! readOnly is now an optional Property Added in a programmatic header and <hr> per design doc adding test for button rendering Adding in convertText function Changing SENTENCE_CASE -> Upper Upper. Need to confirm that this is ok, otherwise can create a new case Reverting SentenceCase Creating PascalCase Removing custom title Moving convert text to EditableSection. Applying by default * removing pascal case * fixing doc * Update amundsen_application/api/utils/metadata_utils.py Co-Authored-By: Tamika Tannis <[email protected]> * Update amundsen_application/api/utils/metadata_utils.py Co-Authored-By: Tamika Tannis <[email protected]> * cleaning up * Fixing unit test to have static method * changing to edit-button * Moving tests to test_metadata_utils * Fixing lint * updating the sample image Co-authored-by: Tamika Tannis <[email protected]> * Clean up some tests; Create constant for hardcoded text (#407) * Fix overflow for issue tracking feature (#406) * Fix overflow for issues from issue tracking feature * try using flex properties * use truncated class * add span Co-authored-by: christina stead <[email protected]> Co-authored-by: Daniel <[email protected]> Co-authored-by: samshuster <[email protected]>

verdan · 2020-04-07T00:07:46Z

amundsen_application/static/js/components/TableDetail/index.tsx

@@ -211,9 +213,10 @@ class TableDetail extends React.Component<TableDetailProps & RouteComponentProps
                  </EditableSection>
                </section>
              </section>
-              {data.programmatic_descriptions.length > 0 &&
+              {
+                data.programmatic_descriptions.length > 0 &&


@ttannis @feng-tao same issue with the truthy conditions here as well. if this value is null then this is breaking the UI. 😢

@verdan is this for the case of the quickstart as well? In practice the metadata service is supposed to be returning an empty array/list for this.

We can fix all these cases but I do want to understand if this is either a confirmed quickstart issue, if it is affecting your use of Amundsen at ING, or if you're just calling this out to be addressed in the future. Will help prioritize how soon I go through and look for all these cases that may have been introduced in 2.1.0

@ttannis
I am using the latest version of both frontend and metadata and it's failing.

Two things here:

This is only implemented on the Neo4j proxy in metadata, hence not available in atlas proxy for now, and returning default value.

Table model in amundsen common says the default value of this variable is None. i.e., programmatic_descriptions: Optional[List[ProgrammaticDescription]] = None

However, in my opinion it should be
programmatic_descriptions: Optional[List[ProgrammaticDescription]] = []

or even this will work:
programmatic_descriptions: List[ProgrammaticDescription] = []

let me know if I can be of any help. It seems like the amundsen common change would be an easy solution?

@samshuster I think that would be correct. I was under the impression the metadata service would be returning an empty list, and should have sanity checked the code in the metada library to ensure this was the case.

@verdan thanks for flagging this. Either of you can feel free to fix if you are able to get to it before I can, and we will be sure to get the PR through and create a new release to unblock as soon as possible.

samshuster · 2020-04-09T15:17:11Z

thank you for catching and fixing @verdan and apologies about that.

Clean up some tests; Create constant for hardcoded text

6632486

ttannis changed the title ~~Clean up some tests; Create constant for hardcoded text~~ Clean up some tests for programmatic descriptions feature Mar 17, 2020

ttannis merged commit fa99d9b into master Mar 17, 2020

ttannis deleted the prog_desc_cleanup branch March 18, 2020 00:46

verdan reviewed Apr 7, 2020

View reviewed changes

verdan mentioned this pull request Apr 8, 2020

Fixes the default value of programmatic_descriptions attribute amundsen-io/amundsencommon#29

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Clean up some tests for programmatic descriptions feature #407

Clean up some tests for programmatic descriptions feature #407

ttannis commented Mar 17, 2020 •

edited

Loading

ttannis commented Mar 17, 2020

samshuster commented Mar 17, 2020

verdan Apr 7, 2020

ttannis Apr 7, 2020 •

edited

Loading

verdan Apr 7, 2020

samshuster Apr 7, 2020

ttannis Apr 7, 2020

samshuster commented Apr 9, 2020

Clean up some tests for programmatic descriptions feature #407

Clean up some tests for programmatic descriptions feature #407

Conversation

ttannis commented Mar 17, 2020 • edited Loading

Summary of Changes

Tests

Documentation

CheckList

ttannis commented Mar 17, 2020

samshuster commented Mar 17, 2020

verdan Apr 7, 2020

Choose a reason for hiding this comment

ttannis Apr 7, 2020 • edited Loading

Choose a reason for hiding this comment

verdan Apr 7, 2020

Choose a reason for hiding this comment

samshuster Apr 7, 2020

Choose a reason for hiding this comment

ttannis Apr 7, 2020

Choose a reason for hiding this comment

samshuster commented Apr 9, 2020

ttannis commented Mar 17, 2020 •

edited

Loading

ttannis Apr 7, 2020 •

edited

Loading