It can certainly be done but it requires a rather clumsy workaround because SP content is restricted by default. If you add a blog sub site to your public site, the posts list can be anonymously accessed, but the comments/categories can't. So for what you want you could embed a listing of posts linked to the posts but comments/categories won't open without a log in.
You can use this solution to make any list anonymously accessible: http://anonymous365.codeplex.com/
There are also some articles here that you may find useful - select "Create a public blog site in Office 365" from the drop down.
And here's an example of an SP blog embedded on a public site. to illustrate what's possible