I'm sure

I'm not being funny

Used as a softening preface to a statement that might be taken as offensive or malicious.

I'm not telling you

Used to express that someone doesn't need to know something, typically because it is a secret (sometimes used as a way to attempt to hide someone's wrongdoing); may also be used when someone tries to ask for answers while doing an assignment or examination, or private or confidential information, including passwords.

I'm not the one

A warning to an individual who has greatly offended one's dignity or has acted or spoken insolently.

if I'm honest

to be honest

if I'm not there, start without me

I am not interested in the event and do not plan to attend.

sure enough

Just as one would expect.

to be sure

Certainly, undoubtedly, admittedly, surely.