Skip to content

test: remove pandas version restrictions and fix precision issue#17005

Draft
chalmerlowe wants to merge 4 commits intomainfrom
feat/assess-pandas-skips-16042
Draft

test: remove pandas version restrictions and fix precision issue#17005
chalmerlowe wants to merge 4 commits intomainfrom
feat/assess-pandas-skips-16042

Conversation

@chalmerlowe
Copy link
Copy Markdown
Contributor

@chalmerlowe chalmerlowe commented May 8, 2026

This PR removes the pandas version restrictions from tests that were skipped for Pandas >= 2.0 without a stated reason and enables testing by addressing a precision issue in the tests.

It also adds a catch for ImportError when tqdm tries to import matplotlib when matplotlib is not installed.

It also adds two small ignore pragmas to resolve mypy's struggles with handling optional imports.

This PR partially fixes #16042 (for bigquery).

For completeness, we did an assessment of any remaining test skips and they all appear to be legitimate skips that are applied when a dependency is not installed.

  1. PyArrow
  2. tqdm (Progress Bar)
  3. Pandas
  4. OpenTelemetry
  5. GeoPandas
  6. Shapely

@chalmerlowe chalmerlowe requested review from a team as code owners May 8, 2026 16:51
@chalmerlowe chalmerlowe requested review from tswast and removed request for a team May 8, 2026 16:51
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request removes pandas version restrictions from system and unit tests and updates the pandas Series assertion logic in test_bq_to_arrow_array_w_pandas_timestamp. The reviewer suggests adding check_names=False to the new assert_series_equal call to prevent potential test failures caused by name mismatches, maintaining the original behavior of checking only value equality.

Comment on lines +561 to +562
series = series.astype(roundtrip.dtype)
pandas.testing.assert_series_equal(series, roundtrip)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While pandas.testing.assert_series_equal provides better diagnostic information than Series.equals(), it is stricter and validates the name attribute by default. Since series is initialized without a name (line 557) and roundtrip might have a name assigned during the Arrow-to-Pandas conversion (e.g., from bq_field.name), this assertion may fail. Adding check_names=False ensures the test continues to focus on value equality, maintaining consistency with the previous equals() check.

Suggested change
series = series.astype(roundtrip.dtype)
pandas.testing.assert_series_equal(series, roundtrip)
series = series.astype(roundtrip.dtype)
pandas.testing.assert_series_equal(series, roundtrip, check_names=False)

@chalmerlowe chalmerlowe marked this pull request as draft May 8, 2026 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[post-migration] test review

2 participants