用strtok分英文单词，怎么空格也被当成单词分出来了？
文档如下：

President Barack Obama delivers remarks to troops at Fort Campbell, Ky., May 6, 2011. Vice President Joe Biden stands at left. (Official White House Photo by Pete Souza)

It was an emotional week, from the relief of the President's announcement that Osama bin Laden was dead to the somber rememberance of the victims in New York City. It ended on a note of gratitude, as the President and Vice President went to Fort Campbell, Kentucky to welcome troops home from a deployment in Afghanistan after a private meeting honoring the military and intelligence professionals involved in the operation against bin Laden.

The Vice President:


在Pete Souza后面分出来了两个空格的单词，
不知道是空格，还是换行符：
分出的单词：Pete
分出的单词：Souza
分出的单词：
分出的单词：
分出的单词：It

strtok里面过滤的单词已经设置成：char seps[] = ".?!';: _,\"-\\\\\\/\t\n[]<>—)(“’…” "

不知道应该修改哪里



[解决办法]
把 \r 也加进去试试。
windows下，换行是\r\n
[解决办法]
printf("%02x\n",(unsigned char)token[0]);//显示token[0]的16进制字节值
[解决办法]
万一文本里面包含'\xa1\xa1'的话……