This is an addition to my post Convenient SSH on Linux cause I was suprised finding that KDE is run from ssh-agent per default. 

# ps aux | grep ssh-agent
userx    6784  0.0  0.0   4480   816 ?        Ss   19:02   0:00 /usr/bin/ssh-agent /usr/bin/gpg-agent --daemon --sh --write-env-file=/home/userx/.gnupg/gpg-agent-info-myhome /usr/bin/startkde

So, first, how can you disable this?

In /etc/X11/Xsession.options :

# cat /etc/X11/Xsession.options
# See Xsession.options(5) for an explanation of the available options.
allow-failsafe
allow-user-resources
allow-user-xsession
use-ssh-agent
use-session-dbus

you can disable use-ssh-agent. This option will be used by
/etc/X11/Xsession.d/90x11-common_ssh-agent
see there for details.

If you want to use the running ssh-agent you can add your keys on startup by putting this script to .kde/Autostart

#!/bin/sh
# set SSH_ASKPASS if not set elsewhere
# export SSH_ASKPASS=/usr/bin/ssh-askpass
for f in /home/userx/.ssh/mykeys/*_dsa ; do ssh-add $f </dev/null ; done ;

Also check the related ubuntu bugs