Apache Hadoop - TLS and SSL Notes
Overview
I’ve collected notes on TLS/SSL for a number of years now. Most of them are related to Apache Hadoop, but others are more general. I was consulting when the POODLE and Heartbleed vulnerabilities were released. Below is a collection of TLS/SSL related references. No guarantee they are up to date but it helps to have references in one place.
TLS/SSL General
- Great explaination of TLS/SSL: http://www.zytrax.com/tech/survival/ssl.html
- SSL Linux certificate location: http://serverfault.com/questions/62496/sslcertificatelocationonunixlinux
- SSL vs TLS: http://security.stackexchange.com/questions/5126/whatsthedifferencebetweenssltlsandhttps
Certificate Types
- http://unmitigatedrisk.com/?p=381
- http://www.cloudera.com/content/cloudera/en/documentation/core/latest/topics/cm_sg_guide_ssl_certs.html
Generating Certificates
- https://www.sslshopper.com/article-most-common-openssl-commands.html
- https://support.ssl.com/Knowledgebase/Article/View/19/0/der-vs-crt-vs-cer-vs-pem-certificates-and-how-to-convert-them
Existing Certificate and Key to JKS
openssl pkcs12 ‐export ‐in abc.crt ‐inkey abc.key ‐out abc.p12
keytool ‐importkeystore ‐srckeystore abc.p12 \
‐srcstoretype PKCS12 \
‐destkeystore abc.jks \
‐deststoretype JKS
Trusting CA Certificates
OpenSSL
update‐ca‐trust force‐enable
cp CERT.pem /etc/pki/tls/source/anchors/
update‐ca‐trust extract
OpenLDAP
vi /etc/openldap/ldap.conf
...
TLS_CAFILE /etc/pki/
# Comment out TLS_CERTDIR
...
Java
/usr/java/JAVA_VERSION/jre/lib/security/cacerts
/etc/pki/ca‐trust/extracted/java/cacerts
POODLE SSLv3
What is POODLE?
- https://poodle.io/servers.html
- https://www.openssl.org/docs/apps/ciphers.html#SSLv3.0ciphersuites
Testing for POODLE
# Requires a relatively recent version of openssl installed
openssl s_client ‐connect HOST:PORT ‐ssl3
# ‐tls1 ‐tls1_1 ‐tls1_2
curl ‐v3 ‐i ‐X HEAD https://HOST:PORT
Configuring Hadoop for Cipher Suites and Protocols
Each Hadoop component must be configured or have the proper version to disable certain SSL protocols and versions.
Ambari
- https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.0.1/configuring-advanced-security-options-for-ambari/content/ambari_sec_optional_configure_ciphers_and_protocols_for_ambari_server.html
security.server.disabled.ciphers=TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA
security.server.disabled.protocols=SSL|SSLv2|SSLv3
Hadoop
- https://issues.apache.org/jira/browse/HADOOP-11243
- Hadoop 2.5.2 + 2.6 Patches SSLFactory for TLSv1
hadoop.ssl.enabled.protocols=TLSv1
- (JDK6 can use TLSv1, JDK7+ can use TLSv1,TLSv1.1,TLSv1.2)
- https://issues.apache.org/jira/browse/HADOOP-11218
- Hadoop 2.8 Patches SSLFactory for TLSv1.1 and TLSv1.2
- Java 6 doesn’t support TLSv1.1+. Requires Java 7.
- https://issues.apache.org/jira/browse/HADOOP-11260
- Hadoop 2.5.2 + 2.6 Patches Jetty to disable SSLv3
HTTPFS
- https://issues.apache.org/jira/browse/HDFS-7274
- Hadoop 2.5.2 + 2.6 Disables SSLv3 in HTTPFS
Hive
- https://issues.apache.org/jira/browse/HIVE-8675
- Hive 0.14 Removes SSLv3 from supported protocols
hive.ssl.protocol.blacklist
- https://issues.apache.org/jira/browse/HIVE-8827
- Hive 1.0 Adds
SSLv2Hello
back to supported protocols hive.ssl.protocol.blacklist=SSLv2,SSLv3
- Hive 1.0 Adds
Oozie
- https://issues.apache.org/jira/browse/OOZIE-2034
- Oozie 4.1.0 Disable SSLv3
- https://issues.apache.org/jira/browse/OOZIE-2037
- Add support for TLSv1.1 and TLSv1.2
- Java 6 doesn’t support TLSv1.1+. Requires Java 7. Depends on OOZIE2036
Flume
- https://issues.apache.org/jira/browse/FLUME-2520
- Flume 1.5.1 HTTPSource disable SSLv3
Hue
- https://issues.cloudera.org/browse/HUE-2438
- Hue 3.8 Disable SSLv3
- line 1670 of
/usr/lib/hue/desktop/core/src/desktop/lib/wsgiserver.py
ctx.set_options(SSL.OP_NO_SSLv2 | SSL.OP_NO_SSLv3)
ssl_cipher_list = "DEFAULT:!aNULL:!eNULL:!LOW:!EXPORT:!SSLv2"
(default)
Ranger
- https://issues.apache.org/jira/browse/RANGER-158
- Ranger 0.4.0 Ranger Admin and User Authentication disable SSLv3
Knox
- https://issues.apache.org/jira/browse/KNOX-455
- Knox 0.5.0 Disable SSLv3
ssl.exclude.protocols
Storm
- https://issues.apache.org/jira/browse/STORM-640
- Storm 0.10.0 Disable SSLv3