The growth and spread of breast cancer are influenced by the composition and structural properties of collagen in the extracellular matrix of tumors. Straight alignment of collagen has been attributed to tumor cell migration, which is correlated with tumor progression and metastasis in breast cancer. Thus, there is a need to characterize collagen alignment to study its value as a prognostic biomarker. We present a framework to characterize the curliness of collagen fibers in breast cancer images from DUET (DUal-mode Emission and Transmission) studies on hematoxylin and eosin (H&E) stained tissue samples. Our novel approach highlights the characteristic fiber gradients using a standard ridge detection method before feeding into the convolutional neural network. Experiments were performed on patches of breast cancer images containing straight or curly collagen. The proposed approach outperforms in terms of area under the curve against transfer learning methods trained directly on the original patches. We also explore a feature fusion strategy to combine feature representations of both the original patches and their ridge filter responses.