Provider Corner
Replies: 6 comments 13 replies
-
This most likely means that your cipher does not have a "get_params" function defined for it. I actually think this is a borderline bug in OpenSSL. IMO, it should not fail in this case - get_params really ought to be an optional thing. Probably just an empty "get_params" function that doesn't actually know about any params would be sufficient to satisfy it (although I've not tried it). Internally when OpenSSL fetches a cipher, libcrypto will cache a number of constants related to the cipher such as the block size, the IV length, whether it supports AEAD, etc. It uses the "get_params" function to obtain those values.
You should make sure you've read the man page for writing provider based ciphers: https://www.openssl.org/docs/man3.0/man7/provider-cipher.html There is a hint on that page as to what parameters you might want to consider supporting in the "Cipher Parameters" section:
Looking at the relevant section on the The Vigenere cipher that @levitte has provided here just supports 2 params ("blocksize" and "keylen"):
It does. |
Beta Was this translation helpful? Give feedback.
-
It does not. I'll be working on that today as best I can. I really understand the nature and need for the provider functions. I have been having problems with the documentation for it through. I appreciate the help on this. My hope is that the get_params issue will resolve the other reported errors. Then I can get to work on all of the other errors I'm sure to introduce :) |
Beta Was this translation helpful? Give feedback.
-
|
If you have any suggestions for changes we can make to the docs to make them more helpful, I'd like to hear them. Even better if you have a PR to make those changes! Sometimes as developers it is difficult to view things from a new user perspective :-) |
Beta Was this translation helpful? Give feedback.
-
|
Writing documentation is always a huge problem for all of us. When I'm trying to write new docs for HPN-SSH it's always frustrating because I'm too close to the work. Anyway, for my own selfish reasons I'd really love it if there was a tutorial on developing providers. Richard's vigenere sample is a great resource and I may, if I can get this working and enough brain cells working, try writing one based on it. If nothing else maybe that can act as a framework for further writing. |
Beta Was this translation helpful? Give feedback.
-
I raised an issue about this. |
Beta Was this translation helpful? Give feedback.
-
|
Also, I hate to bring this up - while the original code (with the _meth_new) compiled with deprecated warnings under OSSL3 the end result was a null cipher. I only figured that out because connections between 1.x and 3.0 compiled code failed. After running a tcpdump I could clearly see the data being sent en clear. So I had a situation where things seemed to work but were, in fact, failing miserably. I'm guessing my situation is a corner case but to me I feel that "unexpected behaviour" might be a bit of an understatement. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, this is the issue you raised previously right? It's on my radar to fix. |
Beta Was this translation helpful? Give feedback.
-
|
Ah, okay, I didn't make the connection. Sorry to bring it up again. |
Beta Was this translation helpful? Give feedback.
-
|
Gotta love it when other OpenSSL devs are paying attention here. I wasn't expecting that 😉 |
Beta Was this translation helpful? Give feedback.
-
|
Well I kinda pointed @rapier1 in this direction in this first place so I was looking out for the discussion: |
Beta Was this translation helpful? Give feedback.
-
|
Ah! |
Beta Was this translation helpful? Give feedback.
-
|
It is pretty cool and I really do appreciate the help both of you have provided. |
Beta Was this translation helpful? Give feedback.
-
|
We'll have to see whether we actually need to make a get_params mandatory. That wasn't the plan, though... As an experiment, would it make sense to you to provide a get_params function, like the vigenere does, and even though it might turn out unnecessary in the long run? At a minimum, you could have it simply |
Beta Was this translation helpful? Give feedback.
-
|
I made that very experiment myself just now, i.e. this in the vigenere provider: diff --git a/vigenere.c b/vigenere.c
index 394142b..c047b51 100644
--- a/vigenere.c
+++ b/vigenere.c
@@ -271,7 +271,7 @@ static int vigenere_get_params(OSSL_PARAM params[])
{
OSSL_PARAM *p;
int ok = 1;
-
+#if 0
for (p = params; p->key != NULL; p++) {
if (strcasecmp(p->key, "blocksize") == 0)
if (provnum_set_size_t(p, 1) < 0) {
@@ -295,6 +295,7 @@ static int vigenere_get_params(OSSL_PARAM params[])
}
}
}
+#endif
return ok;
}
The result isn't a pretty picture, unfortunately. Its $ (cd _build; ctest -VV)
...
3: hex string is too long, ignoring excess
3: error writing output file
3: 40A799DA977F0000:error:030000BD:digital envelope routines:EVP_EncryptUpdate:update error:../crypto/evp/evp_enc.c:630:
3: hex string is too long, ignoring excess
3: bad decrypt
3: 40D76859DC7F0000:error:030000BC:digital envelope routines:EVP_DecryptFinal_ex:final error:../crypto/evp/evp_enc.c:913:
3: # Failed test 'encrypting with 'openssl enc -provider vigenere -e -vigenere -K 0123456789ABCDEF0123456789ABCDEF -in 01-cipher-count.txt''
3: # at /home/levitte/gitwrk/github.com/provider-corner/vigenere/t/01-cipher.t line 42.
3: # +-----+----+-------+
3: # | GOT | OP | CHECK |
3: # +-----+----+-------+
3: # | 256 | eq | 0 |
3: # +-----+----+-------+
3: # Failed test 'encryption result'
3: # at /home/levitte/gitwrk/github.com/provider-corner/vigenere/t/01-cipher.t line 43.
3: # +-----+----+---------------------------------------------------------+
3: # | GOT | OP | CHECK |
3: # +-----+----+---------------------------------------------------------+
3: # | | eq | 558baa87fa2036526c43a7d9f8223b0f6792bd87f3203a5f7443b4d |
3: # | | | dee1ded63698865d3ea25460f6592ac |
3: # +-----+----+---------------------------------------------------------+
3: # Failed test 'decrypting with 'openssl enc -provider vigenere -e -vigenere -K 0123456789ABCDEF0123456789ABCDEF -in 01-cipher-count.txt''
3: # at /home/levitte/gitwrk/github.com/provider-corner/vigenere/t/01-cipher.t line 53.
3: # +-----+----+-------+
3: # | GOT | OP | CHECK |
3: # +-----+----+-------+
3: # | 256 | eq | 0 |
3: # +-----+----+-------+
3: # Failed test 'decryption result'
3: # at /home/levitte/gitwrk/github.com/provider-corner/vigenere/t/01-cipher.t line 54.
3: # +-----+----+---------------------------------------------+
3: # | GOT | OP | CHECK |
3: # +-----+----+---------------------------------------------+
3: # | | eq | The quick brown fox jumps over the lazy dog |
3: # +-----+----+---------------------------------------------+
3:
3: # Failed test 'plain vigenere, 128 bit key'
3: # at /home/levitte/gitwrk/github.com/provider-corner/vigenere/t/01-cipher.t line 10.
...The OpenSSL error message that's seen there happens here (yes, some code has moved, so not exact line number): https://github.com/openssl/openssl/blob/openssl-3.0/crypto/evp/evp_enc.c#L632-L637 |
Beta Was this translation helpful? Give feedback.
-
|
Ah, so presumably block size is 0 and it doesn't like it? A block size of 0 is probably not a sensible default (1 would be better) but without knowing the "real" blocksize we will not be able to pad correctly. |
Beta Was this translation helpful? Give feedback.
-
|
I was wondering if libcrypto really must care... but yeah, unless the cipher is really a stream cipher, I guess it must. |
Beta Was this translation helpful? Give feedback.
-
|
Hey, I want to thank the both of you I have the provider working. I don't think the code is as efficient/compact as it could be but I got a lot out of this discussion. I know I'm still missing quite a bit (the setting and getting of parameters is still a bit mysterious to me) but I'm not mad at where I am now. After I clean up the code a bit more I'll make it available. |
Beta Was this translation helpful? Give feedback.
-
|
No problem! Glad you got it working. |
Beta Was this translation helpful? Give feedback.
-
|
You're very welcome |
Beta Was this translation helpful? Give feedback.
-
Richard,
Thank you for opening up this discussion and being willing to answer questions. I'll do my very best to respect your time and please feel free to tell me to back off when necessary.
I'd like to start with some background if that's okay. Quite a few years ago the team I was leading developed a way to parallelize the AES-CTR cipher specifically for ue in our version of SSH (called hpn-ssh). This was prior to the widespread deployment of AES-NI so any improvement we could get was important for our needs. Essentially what we did was use multiple threads to fill a series of queues with keystream data produced via the EVP interface. As en/decipher requests came in on the main thread we could pull in the corresponding block from the keystream queue, xor it, and move on. We modified the existing cipher with the following:
Obviously, none of that works under OpenSSL 3 (in fact, it fails in a weirdly silent way and produces a NULL cipher).
Anyway, that's the background. I've made a couple of passes at building a provider but I'm running into some problems. In my case it turns out that I can load the provider but when I try to load the cipher itself I get a number of errors:
Now, I'm guessing that I need to have some way to get and set parameters. I'm at a loss as to what these parameters should be and the order in which these should be inserted into any particular table. I can probably figure that out from vigenere cipher but my concern is that every provider I've seen so far seems to be built as a standalone library that is loaded into OpenSSL. I had guessed that using OSSL_PROVIDER_add_builtin() would allow me to keep the definition of the provider as part of the application code instead of a stand alone library but I'm not sure about that. In any case, the provider definition itself seems to load as OSSL_PROVIDER_available() returns 1 when I query the provider name.
Unfortunately, when I try to fetch the cipher routine with EVP_CIPHER_fetch(NULL, "aes_ctr_mt", "provider=hpnssh") it fails.
Are there any steps that need to be taken between loading the provider and loading a specific method from the provider? The provider query function seems to be returning the array with the function pointers but I can't help but think there are some intervening steps that need to take place that I'm not aware of. As a note I started by trying to build off of the example in the provider manpage.
I apologize for this being all over the place. My brain is working at half capacity due to a recent loss in my family.
Beta Was this translation helpful? Give feedback.
All reactions