If you had installed NVidia driver with NVidia's "binary blob" installer, then it had placed files somewhere.
The kmod-nvidia from ELRepo does not know about unmanaged files and is thus likely to overwrite unmanaged files.
Replacing (some) files silently is quite different from managed uninstall of old package & managed install of new package.
NVidia does have its own Yum-repositories too. They are mainly for CUDA, but do contain drivers too -- as RPM packages.
Just like ELRepo's RPM packages, the NVidia's RPM packages require specific version of kernel.
I bet they do not support "kernel-ml" either.