I was facing similar problem with Redis python library. I follow the same documentation instructions which you mentioned in your second link.
here is example snippet for your reference :
Create new directory MyPythonLambda and put MyPythonLambda.py in the same.
Assume MyPythonLambda/MyPythonLambda.py is main lambda containing handler.
cd MyPythonLambda/ pip install redis -t . zip -r MyPythonLambda.zip *
Upload/import zip in lambda creation from S3 or your local file system.
I think you need to create zip file in similar way containing your python mysql library.