Eng
lish
Kontakt
Išči
Meni
Tečaji za odrasle
Izpiti
Knjige
Za otroke
Na tujih univerzah
Seminar SJLK
Simpozij obdobja
Izobraževanja za učitelje
34. Obdobja - 1. del:
Pojdi na
1
2-3
4-5
6-7
8-9
10-11
12-13
14-15
16-17
18-19
20-21
22-23
24-25
26-27
28-29
30-31
32-33
34-35
36-37
38-39
40-41
42-43
44-45
46-47
48-49
50-51
52-53
54-55
56-57
58-59
60-61
62-63
64-65
66-67
68-69
70-71
72-73
74-75
76-77
78-79
80-81
82-83
84-85
86-87
88-89
90-91
92-93
94-95
96-97
98-99
100-101
102-103
104-105
106-107
108-109
110-111
112-113
114-115
116-117
118-119
120-121
122-123
124-125
126-127
128-129
130-131
132-133
134-135
136-137
138-139
140-141
142-143
144-145
146-147
148-149
150-151
152-153
154-155
156-157
158-159
160-161
162-163
164-165
166-167
168-169
170-171
172-173
174-175
176-177
178-179
180-181
182-183
184-185
186-187
188-189
190-191
192-193
194-195
196-197
198-199
200-201
202-203
204-205
206-207
208-209
210-211
212-213
214-215
216-217
218-219
220-221
222-223
224-225
226-227
228-229
230-231
232-233
234-235
236-237
238-239
240-241
242-243
244-245
246-247
248-249
250-251
252-253
254-255
256-257
258-259
260-261
262-263
264-265
266-267
268-269
270-271
272-273
274-275
276-277
278-279
280-281
282-283
284-285
286-287
288-289
290-291
292-293
294-295
296-297
298-299
300-301
302-303
304-305
306-307
308-309
310-311
312-313
314-315
316-317
318-319
320-321
322-323
324-325
326-327
328-329
330-331
332-333
334-335
336-337
338-339
340-341
342-343
344-345
346-347
348-349
350-351
352-353
354-355
356-357
358-359
360-361
362-363
364-365
366-367
368-369
370-371
372-373
374-375
376-377
378-379
380-381
382-383
384-385
386-387
388-389
390-391
392-393
394-395
396-397
398-399
400-401
402-403
404-405
406-407
408-409
410-411
412-413
414-415
416-417
418-419
420-421
422-423
424-425
426-427
428-429
430-431
432
Simpozij OBDOBJA34 language elements (7% in Croatian, 8% in Serbian) are very similar to Slovene (Cro: njomnjom, Ser: alooo). 4 Automatic prediction of standardness level For the automatic prediction of the level of standardness we trained a regression model for each language (Slovene, Croatian and Serbian) and each dimension of standardness (technical and linguistic) on the manually annotated tweets. We used a support-vector machine regressor with an RBF kernel, as implemented in the scikit- learn toolkit (Pedregosa et al. 2011). We represented the content of each tweet through 29 independent variables. Most were string-based (punctuation, vowel-consonant ratio, the ratio of alphabet characters, etc.), some were token-based (e.g. the ratio of short words) and a few of the variables lexicon-based (i.e. they relied on an external information source, such as a lexicon of standard language, which enabled us to determine the out-of-vocabulary ratio of all words, only short words, etc.). Theresults of automatic prediction of standardness level for the three sub-corpora are given in Table 1. They confirm our early intuition that Twitter data are quite standard, with 67–73% of the corpus classified as score 1. Slovene and Croatian tweets are particularly standard, in all likelihood because in these languages Twitter is predominantly used by official accounts for information dissemination. At the other end of the spectrum, Slovene and Croatian also have a larger share of very nonstandard tweets than Serbian, consistent with the results of manual analysis, and confirming that nonstandard orthography prevails in Slovene (and to a lesser degree Croatian), whereas nonstandard lexis is characteristic of Serbian, most likely reflect- ing the much younger profile of Serbian Twitter users. Table 1: Distribution of standardness by language Language Score 1 Score 2 Score 3 Slovene 70% 23% 7% Croatian 73% 21% 6% Serbian 67% 30% 3% Weevaluated the results using mean absolute error, which showed that the auto- maticestimateofthelinguisticstandardnesswasonaverage0.41pointsincorrectwith respect to manual annotation for Slovene, 0.44 for Serbian and 0.46 for Croatian. The best score was obtained on Slovene data as the Sloleks1 lexicon that was used to extract some features was significantly larger than those for Croatian (Apertium2) and Serbian (Wikipedia and news-corpora based lexicon). The results for the technical 1 http://www.slovenscina.eu/sloleks 2 https://www.apertium.org 229