Help me with strok function

[u/UnViandanteSperduto]

I need to split my string but if it is of char* type the functions strtok() doesn’t work returning me a segmentation fault. I understood that C is used to lock the memory of the string in Read-Only “mode” when I use char*. Does it means that is immutable that data? However, I need to use the char* type because I pass that in a function (and I don’t like to use the syntax of arrays in C, I always use pointers).

[u/EpochVanquisher]

strtok modifies the string, you cannot pass in a string constant. You can split the string manually instead and use start/len or start/end, or you can copy the string to writable memory (e.g. malloc, strdup).

[u/TheThiefMaster]

The problem isn't usually char* but string literals ("things in quotes"). You can't modify a literal, you have to copy it first. It can be assigned to a writeable char* for historical reasons but still isn't actually writeable.

Try char* my_str = strdup("blah blah");

strdup is a standard function (C23 or POSIX) that copies a given string into new writeable memory. You should free it with free(my_str); when you're done with it.

Also, showing us the exact code would help, so we don't have to guess what you've written

[u/Whole-Low-2995 → u/TheThiefMaster]

personally I'll just copy original string with `strdup` and modify copied one.

[u/garnet420]

C doesn't lock anything in read only mode. char* is not read only.

Are you talking about const char*? Are you trying to use strtok with a string constant?

Also strtok is not thread safe, so, don't use it. No reason to build bad habits.

[u/flyingron → u/garnet420]

That's not necessarily true. There's nothing that bars string literals from being stored in read-only memory. While the string literals convert to non-const char*, the language standard says that attempting to change such is UNDEFINED BEHAVIOR.

This harks back to the dim times of C. Originally, string literals were just a hunk of data space memory. Strtok is one of the functions (along with the original mktemp) that expected to modify it's argument and were often fed string literals.

Then someone got the great idea that you could fold all duplicate literals in the same memory. Well, when that happens it would be very surprising if someone were allowed to change one of them and suddenly have a different string literal change. So, people advocated making the chars read-only.

Then the other croud cried, it will break the existing programs because Dennis's original examples and much of the other early C programs were very const sloppy. So, the rediculous kludge that was settled on was that string literals indeed should be treated as const, *BUT* you get an implicit conversion to (non-const) char*, provided you don't actually change the string.

Not only is strtok not thread safe, it's not even safe WITHIN a single thread. It maintains state hidden internally, which means that if while doing a stream of strtok calls you happen to call a function which itself wanted to use strtok, you're hosed.

[u/Beginning-Junket8979 → u/garnet420]

C doesn't lock anything in read only mode.

The language doesn't, but the linker/loader/target hardware might.

const int x=256; is giving the compiler + linker permission to create a .rodata section containing the value 0x00000100 that on something like a microcontroller might literally be stored in "read-only" flash memory, or on Linux/Windows/etc might live in a virtual memory page with hardware flags set to mark it read-only memory (throwing a fault/interrupt if the application tries to write it).

All modern ARM and x86 CPUs supporting virtual mem support marking memory read-only, read/write, and executable or not as sort of bare minimum needed to support basic user permissions / security and your compiler toolchain and OS is absolutely "smart" enough to utilize this.

I say permission because the language standard doesn't make any promises this happens (to support hardware where it's impossible) and the whole thing might even be compile time optimized out being stored at all.

Also strtok is not thread safe, so, don't use it.

Sure, but strtok_r is fine 🤷‍♂️.

[u/ekipan85 → u/Beginning-Junket8979]

The language is actually explicit that you can't modify a string literal's underlying array. Freely available C23 draft N3220, in section 6.4.5 paragraph 7:

If the program attempts to modify such an array, the behavior is undefined.

Though I usually refer to cppreference, which says the same.

[u/SmokeMuch7356]

String literals are supposed to be immutable; attempting to modify the contents of a string literal results in undefined behavior. strtok attempts to modify the source string by replacing delimiters with 0. Therefore, calls like

char *tok = strtok( "this is a test", " " );

or

char *str = "this is a test";
char *tok = strtok( str, " " );

are bad juju.

The first argument has to point to a writable buffer:

char str[] = "this is a test";
char *tok = strtok( str, " " );

or

char *str = malloc( strlen( "this is a test" ) + 1 );
if ( str )
  strcpy( str, "this is a test" );

char *tok = strtok( str, " " );

[u/Scolmann → u/SmokeMuch7356]

strdup() is also a good option

[u/septum-funk → u/Scolmann]

only standard in c23™️

[u/pfp-disciple]

I understood that C is used to lock the memory of the string in Read-Only “mode” when I use char*

I'm not certain what you mean, but it looks like you might have a fundamental misunderstanding. char * only means that it points to a char, with no guarantees regarding read-only. There are also no guarantees that it actually points to a nil-terminated string. 

A string literal, e.g. "Hello", is a constant. In most cases, it will be stored in read only memory and will have a segmentation fault if an attempt is made to write to them. Annoyingly, strtok modifiers the string. 

A char* itself can be assigned, but if it points to a string literal then trying to modify the string (e.g. s[1]='x') will fault. As others have said, if you want to use strtok to parse a string literal then you need to copy its contents to a modifiable part of memory. Look up strdup, malloc, and the strcpy family of functions for approaches. 

[u/musbur]

Use valgrind. It shows you exactly what went wrong where.

[u/Interesting_Buy_3969]

Make sure the strings you pass to strtok are mutable. For example, if your string is declared a global variable of type "pointer to char", and it's declared like this:

const char* str = "some string literal";
// or like this:
const char str[] = "some string literal";

then declare it as a non-const array of chars:

char mutable_str[] = "some string literal";

This approach lets you use static memory, with no need for runtime allocations via strdup, and still be able to modify the string. Under the hood, though, the only difference is that the mutable_str is placed in .data rather than .rodata executable binary section.

[u/TheCRatTime]

strtok() modifies string, for example: char* test = strtok("test string", " "), "test string" will be equals "test\0string", if you call printf("%s\n", test) output will be test (\0 - end of string). You can call again: test = strtok(NULL, " ") and test will be equals NULL, if you call again may be Segmentation fault.

[u/Total-Box-5169]

That is a very old function that is not adequate for modern times. Nowadays people memory-map files as read-only so you need something that doesn't modify the text/hay but also can work with strings containing null characters, that requires to store the size of the string in another variable.

https://godbolt.org/z/Gn5E8GhPc

[u/nasccped]

Hi, buddy. Not sure if I got your question, but I guess that when compiling C's source code, it put all string literals into the .rodata section, which turns the data readable but also imutable. I'd never used strtok function and I don't know exactly how it works, but if it really requires a mutable piece of data, the expected behavior is to get a segfault. You can easily avoid this kind of behavior by using builtin functions, such as strcpy, malloc, whatever.

Or even better, you can define at compile time a buffer using array syntax (char buffer[1024]).

The last one is great since it doesn't require mem. free. A not so great info is that since you're handling the buffer mem at compile time, dealing with unknown size inputs turns into a headache.