Yury Bendersky et al.
Replies
0
Voices
1

ClassLoader Interception

The first attempt to break the encrypted bytecode was probably to “intercept” the process of decrypting the bytecode and building the Java class. These attempts have been repeated for almost 20 years, and they are all based on an erroneous premise – “before proceeding to the execution of a java program, the decrypted byte-code must be processed by one of the methods of java ClassLoader
      defineClass(String className, byte[] byte-code, ...)
and if we add the next few lines to defineClass(...) method of the ClassLoader
      File dump = new File(...);
      FileOutputStream out = null;
      try {
         out = new FileOutputStream(dump);
         out.write(byte-code, off, len);
      } catch (IOException ioe) {
         ioe.printStackTrace();
      }
then the dump file will contain the decrypted byte-code ”.
There are at least two errors here. First, the custom ClassLoader is NOT obliged to call the method
      defineClass(String className, byte[] byte-code, ...)
as all defineClass(...) methods ultimately refer to corresponding native methods
      native defineClass0(String name, byte[] b, int off, int len, ProtectionDomain pd);
      native defineClass1(String name, byte[] b, int off, int len, ..., String source);
      …
where the transformation of the byte array into the Java class takes place. My custom ClassLoader refers directly to native methods bypassing Java methods defineClass(...)
Below is an example of code that cannot be intercepted.
      public class CustomClassLoader extends ClassLoader {
         synchronized Class<?> load(String className, byte[] b) {
            ...
            try {
               Class<?> classLoader = Class.forName("java.lang.ClassLoader");
               if (java.version >= 11) {
                  ...
                  Method method = classLoader.getDeclaredMethod("defineClass1", argTypes);
                  Class<?> klass = (Class<?>) method.invoke(this, argVals);
                  ...
                  return klass;
               } else {
                  ...
                  Method method = classLoader.getDeclaredMethod("defineClass0", argTypes);
                  ...
                  return klass;
               }
            } catch (Exception e) {
               return null;
            }
          }
        }
      }
It is impossible to embed interception code into a native method.
The second mistake is that the protected code is not obliged both to work at all and
even to be decrypted in a non-standard environment, in particular with a modified ClassLoader.
Not genuine ClassLoader can be easily detected with a simple parser, derived from [3]
      public class ClassInputStream extends DataInputStream {
         ...
         public ClassInputStream(final InputStream in) {
            super(in);
            input = (DataInputStream) in;
         }
         ...
         private boolean readMethods() {
            try {
              int methodsCount = input.readUnsignedShort();
                final Method[] methods = new Method[methodsCount];
                for (int i = 0; i < methodsCount; i++) {
                   final Method method = new Method();
                   method.read();
                   methods[i] = method;
                }
                for (int i = 0; i < methodsCount; i++) {
                   if (methods[i].name.toString().contains("defineClass")) {
                      defineMethodsString += (methods[i].toString() + "\n");
                   }
                }
            } catch (final IOException e) {
               return false;
            }
            return true;
         }
         ...
         class Method {...}
         …
         public List<Integer> getDefineMethodsList() {
            if (list.size() == 0) {
               getDefineMethodsString();
            }
            return list;
         }
         private String getDefineMethodsString() {
            if (defineMethodsString.length() == 0) {
               readMethods();
            }
            return defineMethodsString;
         }
      }
      public class Main {
         static public void main(String[] args) {
            if (verifyClassLoader()) {
               decryptAndRun();
            }
      }
      public static boolean verifyClassLoader() {
         Map<String, List<Integer>> map = new HashMap<String, List<Integer>>();
         map.put("LINUX_64_ORA_58.0", Arrays.asList( 34, 35, 68, 140, 254 ));
         map.put("LINUX_64_ORA_59.0", Arrays.asList( 34, 35, 68, 140, 254 ));
         map.put("MACOS_64_ORA_58.0", Arrays.asList( 36, 35, 387, 112 ));
         map.put("MACOS_64_ORA_59.0", Arrays.asList( 36, 35, 387, 112 ));
         map.put("WINDOWS_64_ORA_58.0", Arrays.asList( 34, 35, 68, 140, 254 ));
         map.put("WINDOWS_64_ORA_59.0", Arrays.asList( 34, 35, 68, 140, 254 ));
         String sys = System.getProperty("os.name").toUpperCase().split(" ")[0];
         sys += "_" + System.getProperty("sun.arch.data.model") + "_";
         sys += System.getProperty("java.vm.vendor").toUpperCase().substring(0, 3);
         sys += "_" + System.getProperty("java.class.version");
         InputStream is = Main.class.getResourceAsStream("/java/lang/ClassLoader.class");
         try {
            DataInputStream dis = new DataInputStream(is);
            ClassInputStream cis = new ClassInputStream(dis);
            if (map.containsKey(sys)) {
               List<Integer> list = cis.getDefineMethodsList();
               if (Arrays.equals(map.get(sys).toArray(), list.toArray()) == false) {
                  System.err.println("not genuine ClassLoader");
                  return false;
               }
            }
         } catch (Exception e) { return false; }
             return true;
         }
      }

Conclusion
The above reasoning is NOT proof or even just plausible reasoning. The difference between one and the other is well explained in [2].
They are just arguments in favor of some of the facts that we can observe in practice. These facts are
– there is no universal method for absolute cracking the source code;
similarly, there is no absolute protection, unfortunately. Protection of the source code is not the end result but a process. It is a war between attackers and defenders, as a war between virus – and antivirus. Like in any war, the defense has always an advantage. According to Clausewitz[1], the defense side has at least a 3-fold advantage. 
In the following chapters, I will provide examples of other hacking methods and the corresponding protection methods.

  1. Carl von Clausewitz, On War, 1832
  2. George Polya, Mathematics and Plausible Reasoning, 1954
  3. Apache Commons Bytecode Engineering Library (BCEL)

Leave a comment