Hive Udf (Encryption and Decryption) in Java

how to encrypt the data in hive
hive encryption
hive column encryption
aes_encrypt hive
encryption function in hive
spark "encrypt" column
image encryption and decryption in java source code
hive udf for masking

Actually i have written an Hive UDF in Java for Encryption and Decryption. But it has some minor Bug in it. I couldn't find it, someone can Please rectify and suggest me some changes..

Problem:
 When i tried to execute this code using Hive it is showing some 'Null' columns for each
 row.
   Encrypted Ex:  1      fdfsvansjw=
                  NULL   NULL
                  2      adf4vandjw=
                  NULL   NULL

  Actually it has to be displayed without NULL Values.When i tried to decrypt the above
  data it is adding Newline Character '/n' in place of Null.
  Decrypted Ex:  1      AAA
                  /n    /n
                  2      BBB
                  /n     /n
Code for Encryption :
package Encrypt;

import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
import java.security.*;
import org.apache.commons.codec.binary.Base64;
import java.io.*;
import java.security.InvalidAlgorithmParameterException;
import java.security.InvalidKeyException;
import javax.swing.JOptionPane;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import javax.crypto.BadPaddingException;
import javax.crypto.Cipher;
import javax.crypto.IllegalBlockSizeException;
import javax.crypto.NoSuchPaddingException;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

public final class En1 extends UDF {

public Text evaluate(final Text s) throws Exception {
if (s == null) {
 return null;
}
byte[] sharedvector = {
0x01, 0x02, 0x03, 0x05, 0x07, 0x0B, 0x0D, 0x11
};

String EncText = "";
byte[] keyArray = new byte[24];
byte[] temporaryKey;
String key = "developersnotedotcom";
byte[] toEncryptArray = null;

//try
   // {

    toEncryptArray =  s.toString().getBytes("UTF-8");        
    MessageDigest m = MessageDigest.getInstance("MD5");
    temporaryKey = m.digest(key.getBytes("UTF-8"));

    if(temporaryKey.length < 24) // DESede require 24 byte length key
    {
        int index = 0;
        for(int i=temporaryKey.length;i< 24;i++)
        {                   
            keyArray[i] =  temporaryKey[index];
        }
    }        

    Cipher c = Cipher.getInstance("DESede/CBC/PKCS5Padding");            
    c.init(Cipher.ENCRYPT_MODE, new SecretKeySpec(keyArray, "DESede"), new IvParameterSpec(sharedvector));            
    byte[] encrypted = c.doFinal(toEncryptArray);            
    EncText = Base64.encodeBase64String(encrypted);


//  }
   /* catch(NoSuchAlgorithmException | UnsupportedEncodingException | NoSuchPaddingException | InvalidKeyException | InvalidAlgorithmParameterException | IllegalBlockSizeException | BadPaddingException NoEx)
{
    //JOptionPane.showMessageDialog(null, NoEx);
     System.out.println(NoEx);
     System.exit(1);
}*/

return new Text(EncText.toString());        
}

}
Input:
Actual I/p Ex:    1      AAA
                  2      BBB

Encrypted O/p Ex:     1      fdfsvansjw=
                      NULL   NULL
                      2      adf4vandjw=
                      NULL   NULL
Code For Decryption :
package Encrypt;

import org.apache.hadoop.hive.ql.exec.UDF;
import org.apache.hadoop.io.Text;
import org.apache.commons.codec.binary.Base64;
import org.apache.hadoop.hive.ql.exec.FunctionTask;
import java.security.MessageDigest;

import javax.crypto.Cipher;

import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

public final class Dec1 extends UDF {

public Text evaluate(final Text s) {
  if (s == null) {
    return null;
   }
  byte[] sharedvector = {
   0x01, 0x02, 0x03, 0x05, 0x07, 0x0B, 0x0D, 0x11
   };

String RawText = "";
byte[] keyArray = new byte[24];
byte[] temporaryKey;
String key = "developersnotedotcom";
byte[] toEncryptArray = null;

try
  {

    MessageDigest m = MessageDigest.getInstance("MD5");
        temporaryKey = m.digest(key.getBytes("UTF-8"));           

        if(temporaryKey.length < 24) // DESede require 24 byte length key
        {
            int index = 0;
            for(int i=temporaryKey.length;i< 24;i++)
            {                  
                keyArray[i] =  temporaryKey[index];
            }
        }

        Cipher c = Cipher.getInstance("DESede/CBC/PKCS5Padding");
        c.init(Cipher.DECRYPT_MODE, new SecretKeySpec(keyArray, "DESede"), new IvParameterSpec(sharedvector));
        byte[] decrypted = c.doFinal(Base64.decodeBase64(s.toString()));    
        RawText = new String(decrypted, "UTF-8"); 
   }
   catch(Exception NoEx)
    {
    //JOptionPane.showMessageDialog(null, NoEx);
     System.out.println(NoEx + "This is Udf error");
     System.exit(1);
    }

   return new Text(RawText.toString());        
}

}
Input:
Decrypted I/p Ex:     1      fdfsvansjw=
                      NULL   NULL
                      2      adf4vandjw=
                      NULL   NULL

Decrypted o/p Ex:    1      AAA
                     /n     /n
                     2      BBB
                     /n     /n

There should'nt be any Null's or /n when encryption and decryption.
Tried to find out the bug. But can't find out.
Please Help me.

Thanks

The cause is not related to the Hive.

The encrypted string is separated by CRLFs, so you should remove the \r\n at the end of your encryption method: return new Text(EncText.toString().replaceAll("\r|\n", ""));

Hive Udf (Encryption and Decryption) in Java, The cause is not related to the Hive. The encrypted string is separated by CRLFs, so you should remove the \r\n at the end of your encryption� Can you decrypt that stuff with a single static Java method that takes two String args? If yes, then you can use the Hive (undocumented) reflect2 function. – Samson Scharfrichter Jun 20 '17 at 23:18

Thanks @Will Du, Your solution worked for me.

I've implemented this encryption code and ran in to a similar issue. Below change to the return line of Encrypt method did the trick.

Before:

return output;

After:

return output.replaceAll("\r|\n", "");

Column level and UDF Encryption in Hive, For sample Java program on how to encrypt/decrypt using AES algorithms, refer here. Step 2: Create a new Hive table with one column and store the secure key as a value for that column. To retrieve this key from inside the UDF (i.e. encrypt hive and decrypt hive) created in step1, we can use a jdbc connection. @Description (name = " aes_decrypt ", value = " _FUNC_(input binary, key string/binary) - Decrypt input using AES. extended = " AES (Advanced Encryption Standard) algorithm. + " Key lengths of 128, 192 or 256 bits can be used. 192 and 256 bits keys can be used if "

I doubt your UDF evaluates return type Text . Change return type from Text to String as Hive supports following String types :

STRING
VARCHAR (Note: Only available starting with Hive 0.12.0)
CHAR (Note: Only available starting with Hive 0.13.0)

please check the Hive datatypes link

Java AES256 encryption and decryption implementation (Hive udf , Java AES256 encryption and decryption implementation (Hive udf function), Programmer Sought, the best programmer technical posts sharing site. However, it cannot be applied to the selected column and row level in the table of Hive, where most PII that is encrypted is only a part of raw data. In this case, the best solution for now is to use Hive UDF to plug in encryption and decryption implementations on selected columns or partial data in the Hive tables. Sample UDF implementations

Encryption - Apache Hive Essentials, In this case, the best solution for now is to use Hive UDF to plug in encryption and decryption implementations on selected columns or partial data in the Hive� UDF on Hive is written in Java. Function on Spark is written in Scala. Encrypting decrypting functions are basically the same. To compare those two frameworks we run count (to decrypt the whole table and get one value as output).

LanguageManual UDF - Apache Hive, Hive Operators and User-Defined Functions (UDFs) NULL if A or B is NULL, TRUE if any (possibly empty) substring of A matches the Java regular expression B, binary, aes_encrypt(input string/binary, key string/binary), Encrypt input using� We need to create a permanent Hive user defined function (UDF) which uses the AES.jar to decrypt input encrypted string. We can find a sample UDF, here . To write the UDF we also need hive-exec

Solved: Why Hive with UDF is faster than Spark ?, UDF on Hive is written in Java. Function on Spark is written in Scala. Encrypting decrypting functions are basically the same. To compare those� How to easily encrypt and decrypt text in Java Cryptography in java is a separate subject altogether.Probably we will go in detail another time. However there is always a need for simple encryption and decryption process which we can easily incorporate in our code with out many dependency.

Comments
  • Please consider accepting his answer if it answered your question.