Computed tomography (CT) and magnetic resonance imaging (MRI) are one of the most essential imaging techniques in the medical imaging domain. The complementary properties of CT and MRI open a vivid area of research for fusing them to generate a better informative fused image. This paper presents a novel convolutional neural network architecture for fusing CT and MRI images, enhancing the quality and information content of the resultant images. The proposed model emphasizes early edge extraction from CT images and contrast enhancement for MRI images, which significantly improves fusion results, as justified by the improved entropy values. The architecture fuses feature maps from both modalities along the channel dimension, integrating the sharpness of CT edges and the soft tissue details from MRI images. The fusion process is followed by down-sampling the feature maps to reconstruct the final image. Qualitative results reveal enhanced image quality, particularly in edge clarity and brightness, surpassing several state-of-the art methods. Quantitative evaluations, including metrics such as Mutual Information (MI), Structural Similarity Index (SSIM), and Peak Signal to Noise Ratio (PSNR), demonstrate the superior performance of the proposed method, particularly in terms of MI and SSIM. This method provides a significant step forward in medical image fusion, offering potential benefits for clinical applications and diagnostic accuracy.