fix: jsonwriter and recordbatchwriter to respect stats skipping

[jusjosj]

Description

This is an update to JsonWriter and RecordBatchWriter to allow them to write commit log stats information in accordance with delta.dataSkippingNumIndexedCols and
delta.dataSkippingStatsColumns if present on the table. If these fields are unset, then the default behavior of collecting stats for the first 32 columns is preserved

Related Issue(s)

• closes #2982

Tests

Tested by running all unit tests with cargo test as well as followed the instructions in CONTRIBUTING.md

[github-actions]

ACTION NEEDED

delta-rs follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

[codecov]

Codecov Report

Attention: Patch coverage is 90.39146% with 27 lines in your changes missing coverage. Please review.

Project coverage is 72.63%. Comparing base (68d57ef) to head (f5ad8ed).
Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
crates/core/src/writer/record_batch.rs	81.60%	19 Missing and 4 partials ⚠️
crates/core/src/writer/json.rs	97.35%	0 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2989      +/-   ##
==========================================
+ Coverage   72.44%   72.63%   +0.18%     
==========================================
  Files         128      128              
  Lines       40974    41199     +225     
  Branches    40974    41199     +225     
==========================================
+ Hits        29685    29924     +239     
+ Misses       9374     9350      -24     
- Partials     1915     1925      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles

[rtyler]

I am making some modifications to this pull request to make the JsonWriter take a DeltaTable which already has these properties present through the table configuration. This will ensure consistent behavior with other writers, but requires a little refactoring on this under-appreciated code 😄

[jusjosj]

I am making some modifications to this pull request to make the JsonWriter take a DeltaTable which already has these properties present through the table configuration. This will ensure consistent behavior with other writers, but requires a little refactoring on this under-appreciated code 😄

Sounds good thanks! Also I'm using going to be using JsonWriter for a pretty intensive workload, so I'd be happy to chip-in for any improvements/maintenance it might need. If you have anything in mind, feel free to tag me here or on slack.