Electrocardiogram signals (ECG) are vital features to identify a healthy body; diagnosing cardiovascular diseases (CVDs) automatically with resource constraint devices at the edge has caught a significant attention in the rapid growth of smart health-care system. The rapidly developing smart world of IoT demands for low latency, less complex, and energy efficient architectures, and their deployment in real time scenarios. Field Programmable Gate Arrays (FPGAs) is a trading distributed embedded platform proving parallel processing, and flexibility in re-design with its re-configurable advancements. This work demonstrates a spilt 2D convolutional neural network (CNN) performing ECG beat classification on Xilinx’s ZCU 104 FPGA board. The convolution has been carried out depth-wise, and performing a channel-wise addition of the half-split feature maps, so as to minimize the resource utilization by reducing the trainable parameters. A customized FPGA IP has been generated using Vitis High Level Synthesis (HLS) tool, and is implemented on ZCU 104 FPGA for the proposed less complex light-weight model. The design has achieved an accuracy of 98.64% in software, and consumed a total power of 4.177W with a latency of 2.19E8ns in classifying arrhythmia reference to Association for the Advancement of Medical Instrumentation- AAMI EC57 standard.