In this paper we describe our submissions to task I of the GermEval 2018 Shared Task with the goal of identifying offensive language in a set of German tweets. We experiment with two neural architectures and different features. Our submission consists of 3 runs using ensembles of different neural network architectures, each achieving approximately 78 % macro-F1 measure on the last 500 tweets from the training set. The source code for our experiments is publicly available on Github.