Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JsonWriter converts structured parsing error into plain string #2143

Closed
abaerptc opened this issue Jan 29, 2024 · 1 comment · Fixed by #2144
Closed

JsonWriter converts structured parsing error into plain string #2143

abaerptc opened this issue Jan 29, 2024 · 1 comment · Fixed by #2144
Assignees
Labels
bug Something isn't working

Comments

@abaerptc
Copy link
Contributor

Environment

Delta-rs version:
0.16.5 and main branch

Binding:
Rust

Environment:

  • Cloud provider:
  • OS: macOS Ventura 13.2.1
  • Other:

Bug

What happened:
When parsing fails, JsonWriter converts DeltaWriterError::DeltaTable(DeltaTableError::Arrow { source: ArrowError::JsonError(_) }) to a DeltaTableError::Generic(string). This makes it impossible to handle error cases in client code without parsing strings.

What you expected to happen:
JsonWriter should return DeltaTableError::Arrow{ source: ArrowError::JsonError(_) }.

How to reproduce it:
Add the following unit test in json.rs:

    #[tokio::test]
    async fn test_parsing_error() {
        let table_dir = tempfile::tempdir().unwrap();
        let schema = get_delta_schema();
        let path = table_dir.path().to_str().unwrap().to_string();

        let arrow_schema = <ArrowSchema as TryFrom<&StructType>>::try_from(&schema).unwrap();
        let mut writer = JsonWriter::try_new(
            path.clone(),
            Arc::new(arrow_schema),
            Some(vec!["modified".to_string()]),
            None,
        )
        .unwrap();

        let data = serde_json::json!(
            {
                "id" : "A",
                "value": "abc",
                "modified": "2021-02-01"
            }
        );

        let res = writer.write(vec![data]).await;
        // This assertion fails because the result is actually an `Err(DeltaTableError::Generic(string))`
        assert!(matches!(
            res,
            Err(DeltaTableError::Arrow {
                source: ArrowError::JsonError(_)
            })
        ));
    }

More details:

@abaerptc abaerptc added the bug Something isn't working label Jan 29, 2024
@abaerptc
Copy link
Contributor Author

take

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant