Adding new magics in spark kernels

[prajvan]

Is your feature request related to a problem? Please describe.
Currently sparkkernels support some line and cell magics but most of this functionality is developed by community aims to solve use cases around using local python or shell etc, can community release a way to contribute or to extend more magics in local to add more functionality.. im trying to incoporate remote shell as magics in jupyter lab so i can add functionality of remote shell over emr master node for use cases specific to our team by virtue of this i can do all in a single integrated environment.

Describe the solution you'd like
Way to extend magics in spark kernels so additional magics can be added without making changes on the core code as this code can be customised for use case basis

Describe alternatives you've considered
I have tried to override the core code by adding more magics, anyway to extend the core functionality by plugin based architecture can be highly useful for specific use cases