Response.Output.Write() gives you String. Format-style formatted output and the Response.Write() doesn't.
In ASP.NET the Response object is of type HttpResponse and when you say Response. Write you're really saying (basically) HttpContext.Current.Response.Write and calling one of the many overloaded Write methods of HttpResponse.
Response.Write then calls .Write() on it's internal TextWriter object:
public void Write(object obj){ this._writer.Write(obj);}
HttpResponse also has a Property called Output that is of type, yes, TextWriter, so:
public TextWriter get_Output(){ return this._writer; }
Which means you can to the Response whatever a TextWriter will let you. Now, TextWriters support a Write() method ala String.Format, so you can do this:
Response.Output.Write("Scott is {0} at {1:d}", "cool",DateTime.Now);
But internally, of course, this this is happening:
public virtual void Write(string format, params object[] arg)
{
this.Write(string.Format(format, arg));
}