Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java: XAUTOCLAIM #407

Closed
wants to merge 23 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f840580
current progress
cyip10 Jul 3, 2024
6a9158d
update streamBaseCommands example docs
cyip10 Jul 3, 2024
84e2d08
fix documentation
cyip10 Jul 3, 2024
0184ded
add baseclient
cyip10 Jul 3, 2024
3ebc3ae
finished base transaction
cyip10 Jul 3, 2024
6071dd7
Python: add FUNCTION DUMP and FUNCTION RESTORE commands (#1769)
yipin-chen Jul 3, 2024
ffd7a8a
xpending and xclaim binary version (xautoclaim not implemented in Gli…
talxsha Jul 4, 2024
dc974a2
Glide pop commands (#1654)
talxsha Jul 4, 2024
60e3632
support functionStats with GlideString (#1779)
alon-arenberg Jul 4, 2024
4b61e9f
remove using toGlideStringArgs() and use ArgsBuilder instead (#1782)
alon-arenberg Jul 4, 2024
b863832
Python: renaming tests clients (#1780)
shohamazon Jul 4, 2024
5a6911e
Core + Python: Added support in large script requests (#1781)
barshaul Jul 4, 2024
e095599
Update dependency installation and setup instructions for macOS in de…
shohamazon Jul 4, 2024
70d3516
Python: add SORT_RO command. (#1528)
GilboaAWS Jul 4, 2024
2e14feb
Add function list binary output support (#1800)
yulazariy Jul 4, 2024
473df92
randomkey binary support (#1805)
aws-talbenjo Jul 4, 2024
2cf4c93
Python: add tests bytes (#1791)
adarovadya Jul 4, 2024
043ffdc
Python: more renaming and doc fixes (#1804)
shohamazon Jul 4, 2024
b74c3f0
support lcs with GlideString (#1802)
alon-arenberg Jul 4, 2024
250d7ed
Python: Rename ClusterClientConfiguration to GlideClusterClientConfig…
shohamazon Jul 4, 2024
4ccbc7a
fcall bug fix (#1799)
talxsha Jul 4, 2024
7b20bc1
Update README for Valkey GLIDE (#1812)
shohamazon Jul 4, 2024
a8800dd
need to fix build issue
cyip10 Jul 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/java.yml
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ jobs:
- 17
runs-on: ubuntu-latest
container: amazonlinux:latest
timeout-minutes: 15
timeout-minutes: 20
steps:
- name: Install git
run: |
Expand Down
3 changes: 2 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,12 +79,13 @@
* Python: Added DUMP and Restore commands ([#1733](https://github.com/aws/glide-for-redis/pull/1733))
* Java: Added SCAN command ([#1751](https://github.com/aws/glide-for-redis/pull/1751))
* Python: Type migration for entries_read ([#1768](https://github.com/aws/glide-for-redis/pull/1768))

* Python: Added FUNCTION DUMP and FUNCTION RESTORE commands ([#1769](https://github.com/aws/glide-for-redis/pull/1769))

### Breaking Changes
* Node: Update XREAD to return a Map of Map ([#1494](https://github.com/aws/glide-for-redis/pull/1494))
* Node: Rename RedisClient to GlideClient and RedisClusterClient to GlideClusterClient ([#1670](https://github.com/aws/glide-for-redis/pull/1670))
* Python: Rename RedisClient to GlideClient, RedisClusterClient to GlideClusterClient and BaseRedisClient to BaseClient([#1669](https://github.com/aws/glide-for-redis/pull/1669))
* Python: Rename ClusterClientConfiguration to GlideClusterClientConfiguration ([#1806](https://github.com/aws/glide-for-redis/pull/1806))

## 0.4.1 (2024-02-06)

Expand Down
7 changes: 4 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# GLIDE for Redis
General Language Independent Driver for the Enterprise (GLIDE) for Redis, is an AWS-sponsored, open-source Redis client. GLIDE for Redis works with any Redis distribution that adheres to the Redis Serialization Protocol (RESP) specification, including open-source Redis, Amazon ElastiCache for Redis, and Amazon MemoryDB for Redis.
Strategic, mission-critical Redis-based applications have requirements for security, optimized performance, minimal downtime, and observability. GLIDE for Redis is designed to provide a client experience that helps meet these objectives. It is sponsored and supported by AWS, and comes pre-configured with best practices learned from over a decade of operating Redis-compatible services used by hundreds of thousands of customers. To help ensure consistency in development and operations, GLIDE for Redis is implemented using a core driver framework, written in Rust, with extensions made available for each supported programming language. This design ensures that updates easily propagate to each language and reduces overall complexity. In this Preview release, GLIDE for Redis is available for Python and Javascript (Node.js), with support for Java actively under development.
### Valkey GLIDE

Valkey General Language Independent Driver for the Enterprise (GLIDE), is an AWS-sponsored, open-source Valkey client that includes support for open-source Redis 6.2 to 7.2. Valkey GLIDE works with any distribution that adheres to the Redis Serialization Protocol (RESP) specification, including Amazon ElastiCache, and Amazon MemoryDB.
Strategic, mission-critical applications have requirements for security, optimized performance, minimal downtime, and observability. Valkey GLIDE is designed to provide a client experience that helps meet these objectives. It is sponsored and supported by AWS, and comes pre-configured with best practices learned from over a decade of operating RESP-compatible services used by hundreds of thousands of customers. To help ensure consistency in development and operations, Valkey GLIDE is implemented using a core driver framework, written in Rust, with extensions made available for each supported programming language. This design ensures that updates easily propagate to each language and reduces overall complexity. In this Preview release, Valkey GLIDE is available for Python and Java, with support for Javascript (Node.js) actively under development.

## Supported Engine Versions
GLIDE for Redis is API-compatible with the following engine versions:
Expand Down
2 changes: 1 addition & 1 deletion examples/python/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
## Run
To run the example or any other Python application utilizing GLIDE for Redis, activate the virtual environment that created by the 'Build' stage:
To run the example or any other Python application utilizing Valkey GLIDE, activate the virtual environment that created by the 'Build' stage:
```
cd examples/python
pip install -r requirements.txt
Expand Down
4 changes: 2 additions & 2 deletions examples/python/client_example.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ async def test_standalone_client(host: str = "localhost", port: int = 6379):
# When in Redis is in standalone mode, add address of the primary node,
# and any replicas you'd like to be able to read from.
addresses = [NodeAddress(host, port)]
# Check `GlideClientConfiguration/ClusterClientConfiguration` for additional options.
# Check `GlideClientConfiguration/GlideClusterClientConfiguration` for additional options.
config = BaseClientConfiguration(
addresses=addresses,
client_name="test_standalone_client",
Expand All @@ -59,7 +59,7 @@ async def test_standalone_client(host: str = "localhost", port: int = 6379):
async def test_cluster_client(host: str = "localhost", port: int = 6379):
# When in Redis is cluster mode, add address of any nodes, and the client will find all nodes in the cluster.
addresses = [NodeAddress(host, port)]
# Check `GlideClientConfiguration/ClusterClientConfiguration` for additional options.
# Check `GlideClientConfiguration/GlideClusterClientConfiguration` for additional options.
config = BaseClientConfiguration(
addresses=addresses,
client_name="test_cluster_client",
Expand Down
12 changes: 10 additions & 2 deletions glide-core/src/protobuf/redis_request.proto
Original file line number Diff line number Diff line change
Expand Up @@ -261,6 +261,13 @@ message Command {
}
}

// Used for script requests with large keys or args vectors
message ScriptInvocationPointers {
string hash = 1;
optional uint64 keys_pointer = 2;
optional uint64 args_pointer = 3;
}

message ScriptInvocation {
string hash = 1;
repeated bytes keys = 2;
Expand All @@ -285,7 +292,8 @@ message RedisRequest {
Command single_command = 2;
Transaction transaction = 3;
ScriptInvocation script_invocation = 4;
ClusterScan cluster_scan = 5;
ScriptInvocationPointers script_invocation_pointers = 5;
ClusterScan cluster_scan = 6;
}
Routes route = 6;
Routes route = 7;
}
45 changes: 37 additions & 8 deletions glide-core/src/socket_listener.rs
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ use crate::cluster_scan_container::get_cluster_scan_cursor;
use crate::connection_request::ConnectionRequest;
use crate::errors::{error_message, error_type, RequestErrorType};
use crate::redis_request::{
command, redis_request, ClusterScan, Command, RedisRequest, Routes, ScriptInvocation,
SlotTypes, Transaction,
command, redis_request, ClusterScan, Command, RedisRequest, Routes, SlotTypes, Transaction,
};
use crate::response;
use crate::response::Response;
Expand All @@ -17,7 +16,7 @@ use bytes::Bytes;
use directories::BaseDirs;
use dispose::{Disposable, Dispose};
use logger_core::{log_debug, log_error, log_info, log_trace, log_warn};
use protobuf::Message;
use protobuf::{Chars, Message};
use redis::cluster_routing::{
MultipleNodeRoutingInfo, Route, RoutingInfo, SingleNodeRoutingInfo, SlotAddr,
};
Expand Down Expand Up @@ -364,15 +363,24 @@ async fn cluster_scan(cluster_scan: ClusterScan, mut client: Client) -> ClientUs
}

async fn invoke_script(
script: ScriptInvocation,
hash: Chars,
keys: Option<Vec<Bytes>>,
args: Option<Vec<Bytes>>,
mut client: Client,
routing: Option<RoutingInfo>,
) -> ClientUsageResult<Value> {
// convert Vec<bytes> to vec<[u8]>
let keys: Vec<&[u8]> = script.keys.iter().map(|e| e.as_ref()).collect();
let args: Vec<&[u8]> = script.args.iter().map(|e| e.as_ref()).collect();
let keys: Vec<&[u8]> = keys
.as_ref()
.map(|keys| keys.iter().map(|e| e.as_ref()).collect())
.unwrap_or_default();
let args: Vec<&[u8]> = args
.as_ref()
.map(|keys| keys.iter().map(|e| e.as_ref()).collect())
.unwrap_or_default();

client
.invoke_script(&script.hash, &keys, &args, routing)
.invoke_script(&hash, &keys, &args, routing)
.await
.map_err(|err| err.into())
}
Expand Down Expand Up @@ -490,7 +498,28 @@ fn handle_request(request: RedisRequest, client: Client, writer: Rc<Writer>) {
}
redis_request::Command::ScriptInvocation(script) => {
match get_route(request.route.0, None) {
Ok(routes) => invoke_script(script, client, routes).await,
Ok(routes) => {
invoke_script(
script.hash,
Some(script.keys),
Some(script.args),
client,
routes,
)
.await
}
Err(e) => Err(e),
}
}
redis_request::Command::ScriptInvocationPointers(script) => {
let keys = script
.keys_pointer
.map(|pointer| *unsafe { Box::from_raw(pointer as *mut Vec<Bytes>) });
let args = script
.args_pointer
.map(|pointer| *unsafe { Box::from_raw(pointer as *mut Vec<Bytes>) });
match get_route(request.route.0, None) {
Ok(routes) => invoke_script(script.hash, keys, args, client, routes).await,
Err(e) => Err(e),
}
}
Expand Down
9 changes: 7 additions & 2 deletions go/DEVELOPER.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,10 +85,15 @@ brew install go make git gcc pkgconfig protobuf@3 openssl
export PATH="$PATH:$HOME/go/bin"
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
# Check that the protobuf compiler is installed. A minimum version of 3.20.0 is required.
protoc --version
# Check that the Rust compiler is installed
rustc --version
# Verify the Protobuf compiler installation
protoc --version

# If protoc is not found or does not work correctly, update the PATH
echo 'export PATH="/opt/homebrew/opt/protobuf@3/bin:$PATH"' >> /Users/$USER/.bash_profile
source /Users/$USER/.bash_profile
protoc --version
```

#### Building and installation steps
Expand Down
4 changes: 2 additions & 2 deletions java/DEVELOPER.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Developer Guide

This document describes how to set up your development environment to build and test the GLIDE for Redis Java wrapper.
This document describes how to set up your development environment to build and test the Valkey GLIDE Java wrapper.

### Development Overview

The GLIDE for Redis Java wrapper consists of both Java and Rust code. Rust bindings for the Java Native Interface are implemented using [jni-rs](https://github.com/jni-rs/jni-rs), and the Java JAR is built using [Gradle](https://github.com/gradle/gradle). The Java and Rust components communicate using the [protobuf](https://github.com/protocolbuffers/protobuf) protocol.
The Valkey GLIDE Java wrapper consists of both Java and Rust code. Rust bindings for the Java Native Interface are implemented using [jni-rs](https://github.com/jni-rs/jni-rs), and the Java JAR is built using [Gradle](https://github.com/gradle/gradle). The Java and Rust components communicate using the [protobuf](https://github.com/protocolbuffers/protobuf) protocol.

### Build from source

Expand Down
10 changes: 4 additions & 6 deletions java/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,9 @@
# Getting Started - Java Wrapper
## Valkey GLIDE

## Notice: Java Wrapper - Work in Progress
Valkey General Language Independent Driver for the Enterprise (GLIDE), is an AWS-sponsored, open-source Valkey client that includes support for open-source Redis 6.2 to 7.2. Valkey GLIDE works with any distribution that adheres to the Redis Serialization Protocol (RESP) specification, including Amazon ElastiCache, and Amazon MemoryDB.
Strategic, mission-critical applications have requirements for security, optimized performance, minimal downtime, and observability. Valkey GLIDE is designed to provide a client experience that helps meet these objectives. It is sponsored and supported by AWS, and comes pre-configured with best practices learned from over a decade of operating RESP-compatible services used by hundreds of thousands of customers. To help ensure consistency in development and operations, Valkey GLIDE is implemented using a core driver framework, written in Rust, with extensions made available for each supported programming language. This design ensures that updates easily propagate to each language and reduces overall complexity. In this Preview release, Valkey GLIDE is available for Python and Java, with support for Javascript (Node.js) actively under development.

We're excited to share that the Java client is currently in development! However, it's important to note that this client
is a work in progress and is not yet complete or fully tested. Your contributions and feedback are highly encouraged as
we work towards refining and improving this implementation. Thank you for your interest and understanding as we continue
to develop this Java wrapper.
# Getting Started - Java Wrapper

The Java client contains the following parts:

Expand Down
Loading
Loading