if I'm honest

God's honest truth

Unquestionable, absolute truth.

Honest Abe

A forthright and honest person.

honest John

An ordinary, honest, member of the public

I'm not being funny

Used as a softening preface to a statement that might be taken as offensive or malicious.

I'm not telling you

Used to express that someone doesn't need to know something, typically because it is a secret (sometimes used as a way to attempt to hide someone's wrongdoing); may also be used when someone tries to ask for answers while doing an assignment or examination, or private or confidential information, including passwords.

I'm not the one

A warning to an individual who has greatly offended one's dignity or has acted or spoken insolently.

I'm sure

Used as an interjection to indicate denial or contradiction in response to an assertion or accusation.

if I'm not there, start without me

I am not interested in the event and do not plan to attend.