Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HDDS-11947. [Docs] [User Guide] Impala Integration. #7584

Merged
merged 8 commits into from
Jan 11, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions hadoop-hdds/docs/content/integration/Impala.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
title: Impala
weight: 4
menu:
main:
parent: "Application integrations"
---
<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

Starting with version **4.2.0**, Apache Impala provides full support for
querying data stored in **Apache Ozone**. To utilize this functionality,
ensure that your Ozone version is **1.4.0** or later.
errose28 marked this conversation as resolved.
Show resolved Hide resolved

## Supported Access Protocols
errose28 marked this conversation as resolved.
Show resolved Hide resolved

Impala supports the following protocols for accessing Ozone data:

* ofs
* s3a

Note: The o3fs protocol is **NOT** supported by Impala.

## Supported Bucket Types
errose28 marked this conversation as resolved.
Show resolved Hide resolved

Impala is compatible with Ozone buckets configured with either:

* RATIS (Replication)
* Erasure Coding
adoroszlai marked this conversation as resolved.
Show resolved Hide resolved

## Querying Ozone Data with Impala

**If Ozone is configured as the default file system**, you can run Impala
errose28 marked this conversation as resolved.
Show resolved Hide resolved
queries seamlessly without modifications, just as if the file system were
HDFS. For example:

```sql
CREATE TABLE t1 (x INT, s STRING);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CREATE TABLE t1 (x INT, s STRING);
CREATE TABLE t1 (x INT, s STRING);

```

**If Ozone is not the default file system**, you must specify the Ozone path
explicitly using the LOCATION clause. For example:

```sql
CREATE DATABASE d1 LOCATION 'ofs://ozone1/vol1/bucket1/d1.db';
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
CREATE DATABASE d1 LOCATION 'ofs://ozone1/vol1/bucket1/d1.db';
CREATE DATABASE d1 LOCATION 'ofs://ozone1/vol1/bucket1/d1.db';

```

For additional information, consult the Apache Impala User Documentation
[Using Impala with Apache Ozone Storage](https://impala.apache.org/docs/build/html/topics/impala_ozone.html).
26 changes: 26 additions & 0 deletions hadoop-hdds/docs/content/integration/_index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: "Application integrations"
menu:
main:
weight: 5
---
<!---
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->

{{<jumbotron title="Application integration">}}
jojochuang marked this conversation as resolved.
Show resolved Hide resolved
Many applications can be integrated with Ozone through the the Hadoop compatible ofs interface or the S3 interface.
jojochuang marked this conversation as resolved.
Show resolved Hide resolved
{{</jumbotron>}}
Loading