Divers problèmes avec les hashtags

Il y a un correctif au niveau des hashtags dans la prochaine MAJ : #19 (closed)

A voir si cela corrige ces problèmes.

Après update, j'ai pas l'impression de voir du mieux.

Je confirme, l'update n'a rien changé pour les deux problèmes en question.

Toujours visible avec la version 3.1.2

Il semblerait qu'on ait des tâches sidekiq qui ne sont jamais lancées.

Trouvé l'origine du problème : on a restreint les queues sidekiq dans notre fichier compose. Je vais reprendre posément le compose officiel et refactor ça comme je peux.

J'ai ajusté la configuration pour réactiver toutes les queues sidekiq.

Fix via documentation@8a538d2e

Je remarque aussi qu'on a beaucoup de timeouts sur des tâches sidekiq triviales comme de la récupération de miniature pour un lien. Ca pourrait causer de soucis en lien avec ça : https://github.com/tootsuite/mastodon/issues/8342

A priori pas de souci de queues pleines, et la tâche est bien passée à l'instant, pourtant rien n'est mis à jour. Je vais voir si on peut consulter les logs de debug d'une tâche sidekiq.

J'ai queue la tâche avec sidekiq en debug, mais rien dans les logs indiquant qu'elle a été exécutée, je vais voir si elle est supposée debug, mais a priori oui, au moins les accès bdd via l'ORM devraient apparaître.

Bon, j'étudie le comportement de la tâche : https://github.com/tootsuite/mastodon/blob/9014367bd87e07941134259d87f4d2c327ff37d4/app/workers/scheduler/trending_tags_scheduler.rb et https://github.com/tootsuite/mastodon/blob/915f3712ae7ae44c0cbe50c9694c25e3ee87a540/app/models/trending_tags.rb#L26

Il semble que côté redis les sets sont bien là, et pas vides.

127.0.0.1:6379> SMEMBERS trending_tags:used:1587859200
  1) "24471"
  2) "151476"
  3) "159784"
  4) "22826"
  5) "159069"
  6) "165"
  7) "3"
  8) "106942"
  9) "3159"
 10) "5107"
 11) "159791"
 12) "4833"
[...]

D'ailleurs potentiel joli bug dans la façon dont c'est implémenté. Si les serveurs sont pas sur le même fuseau horaire le timestamp utilisé pour stocker les trends du jour va pas être le même entre sidekiq et web. C'est pas notre cas celà dit, donc pas ça.

Déjà pas de souci avec l'exécution de TrendingTags.update!, j'ai mis un peu de debug et voici :

irb(main):001:0> Rails.logger.level=0
=> 0
irb(main):002:0> TrendingTags.update!
  Tag Load (4.7ms)  SELECT "tags".* FROM "tags" WHERE "tags"."id" IN ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21, $22, $23, $24, $25, $26, $27, $28, $29, $30, $3
1, $32, $33, $34, $35, $36, $37, $38, $39, $40, $41, $42, $43, $44, $45, $46, $47, $48, $49, $50, $51, $52, $53, $54, $55, $56, $57, $58, $59, $60, $61, $62, $63, $64, $65, $66, $67, $68, $69, $70, $71, $72, $73,
$74, $75, $76, $77, $78, $79, $80, $81, $82, $83, $84, $85, $86, $87, $88, $89, $90, $91, $92, $93, $94, $95, $96, $97, $98, $99, $100, $101, $102, $103, $104, $105, $106, $107, $108, $109, $110, $111, $112, $113,
 $114, $115, $116, $117, $118, $119, $120, $121, $122, $123, $124, $125, $126, $127, $128, $129, $130, $131, $132, $133, $134, $135, $136, $137, $138, $139, $140, $141, $142, $143, $144, $145, $146, $147, $148, $1
49, $150, $151, $152, $153, $154, $155, $156, $157, $158, $159, $160, $161, $162, $163, $164, $165, $166, $167, $168, $169, $170, $171, $172, $173, $174, $175, $176, $177, $178, $179, $180, $181, $182, $183, $184,
 $185, $186, $187, $188, $189, $190, $191, $192, $193, $194, $195, $196, $197, $198, $199, $200, $201, $202, $203, $204, $205, $206, $207, $208, $209, $210, $211, $212, $213, $214, $215, $216, $217, $218, $219, $2
20, $221, $222, $223, $224, $225, $226, $227, $228, $229, $230, $231, $232, $233, $234, $235, $236, $237, $238, $239, $240, $241, $242, $243, $244, $245, $246, $247, $248, $249, $250, $251, $252, $253, $254, $255,
 $256, $257, $258, $259, $260, $261, $262, $263, $264, $265, $266, $267, $268, $269, $270, $271, $272, $273, $274, $275, $276, $277, $278, $279, $280, $281, $282, $283, $284, $285, $286, $287, $288, $289, $290, $2
91, $292, $293, $294, $295, $296, $297, $298, $299, $300, $301, $302, $303, $304, $305, $306, $307, $308, $309, $310, $311, $312, $313, $314, $315, $316, $317, $318, $319, $320, $321, $322, $323, $324, $325, $326,
 $327, $328, $329, $330, $331, $332, $333)  [["id", 1], ["id", 3], ["id", 14], ["id", 16], ["id", 47], ["id", 49], ["id", 57], ["id", 61], ["id", 69], ["id", 72], ["id", 80], ["id", 88], ["id", 124], ["id", 184],
["id", 199], ["id", 206], ["id", 262], ["id", 276], ["id", 284], ["id", 285], ["id", 286], ["id", 287], ["id", 314], ["id", 341], ["id", 383], ["id", 384], ["id", 386], ["id", 400], ["id", 405], ["id", 429], ["id"
, 450], ["id", 458], ["id", 461], ["id", 475], ["id", 488], ["id", 504], ["id", 510], ["id", 519], ["id", 520], ["id", 580], ["id", 643], ["id", 658], ["id", 671], ["id", 695], ["id", 726], ["id", 741], ["id", 778
], ["id", 824], ["id", 825], ["id", 835], ["id", 839], ["id", 844], ["id", 922], ["id", 928], ["id", 989], ["id", 1008], ["id", 1010], ["id", 1024], ["id", 1119], ["id", 1125], ["id", 1130], ["id", 1131], ["id", 1
221], ["id", 1247], ["id", 1320], ["id", 1322], ["id", 1422], ["id", 1453], ["id", 1479], ["id", 1522], ["id", 1535], ["id", 1596], ["id", 1655], ["id", 1664], ["id", 1668], ["id", 1711], ["id", 1726], ["id", 1938
], ["id", 1956], ["id", 1976], ["id", 1981], ["id", 1985], ["id", 1990], ["id", 2112], ["id", 2117], ["id", 2193], ["id", 2230], ["id", 2270], ["id", 2354], ["id", 2368], ["id", 2403], ["id", 2509], ["id", 2600],
["id", 2643], ["id", 2704], ["id", 2823], ["id", 3067], ["id", 3069], ["id", 3166], ["id", 3217], ["id", 3227], ["id", 3306], ["id", 3309], ["id", 3784], ["id", 3850], ["id", 4371], ["id", 4441], ["id", 4551], ["i
d", 4591], ["id", 4743], ["id", 4836], ["id", 5070], ["id", 5306], ["id", 5686], ["id", 5984], ["id", 6076], ["id", 6088], ["id", 6304], ["id", 6357], ["id", 6522], ["id", 6680], ["id", 6748], ["id", 6882], ["id",
 7028], ["id", 7034], ["id", 7274], ["id", 7678], ["id", 7697], ["id", 7808], ["id", 7906], ["id", 8289], ["id", 8356], ["id", 8405], ["id", 8552], ["id", 8802], ["id", 8812], ["id", 8967], ["id", 9095], ["id", 96
47], ["id", 9649], ["id", 9662], ["id", 9952], ["id", 10303], ["id", 10322], ["id", 10413], ["id", 10985], ["id", 11102], ["id", 11392], ["id", 12464], ["id", 12607], ["id", 12762], ["id", 12804], ["id", 13030], [
"id", 13238], ["id", 14443], ["id", 14602], ["id", 14627], ["id", 14795], ["id", 15043], ["id", 16630], ["id", 17265], ["id", 17770], ["id", 18466], ["id", 18506], ["id", 19794], ["id", 20544], ["id", 21825], ["id
", 22197], ["id", 22656], ["id", 24455], ["id", 24739], ["id", 25704], ["id", 25850], ["id", 25911], ["id", 26157], ["id", 26452], ["id", 26860], ["id", 27040], ["id", 27950], ["id", 28243], ["id", 28577], ["id",
29154], ["id", 29971], ["id", 29973], ["id", 30012], ["id", 30121], ["id", 30436], ["id", 31133], ["id", 31425], ["id", 31521], ["id", 31920], ["id", 31978], ["id", 32716], ["id", 32742], ["id", 33762], ["id", 338
57], ["id", 34908], ["id", 35244], ["id", 35863], ["id", 35958], ["id", 35991], ["id", 36582], ["id", 37854], ["id", 38233], ["id", 39408], ["id", 42218], ["id", 44376], ["id", 44511], ["id", 45546], ["id", 45609]
, ["id", 47507], ["id", 47947], ["id", 52328], ["id", 54374], ["id", 58858], ["id", 60614], ["id", 62004], ["id", 62138], ["id", 68073], ["id", 70580], ["id", 74610], ["id", 75369], ["id", 75410], ["id", 80025], $
"id", 80168], ["id", 81854], ["id", 87442], ["id", 88164], ["id", 88673], ["id", 96870], ["id", 97802], ["id", 100653], ["id", 102715], ["id", 105962], ["id", 111940], ["id", 112478], ["id", 112511], ["id", 112525
], ["id", 114929], ["id", 116027], ["id", 117455], ["id", 120345], ["id", 120511], ["id", 120512], ["id", 134105], ["id", 137349], ["id", 137350], ["id", 137351], ["id", 137497], ["id", 137498], ["id", 141754], ["
id", 142264], ["id", 142266], ["id", 142560], ["id", 143090], ["id", 145265], ["id", 145984], ["id", 148176], ["id", 148557], ["id", 149562], ["id", 149563], ["id", 150503], ["id", 150549], ["id", 151887], ["id",
152436], ["id", 153171], ["id", 153330], ["id", 154298], ["id", 154464], ["id", 154610], ["id", 155351], ["id", 155788], ["id", 156079], ["id", 156100], ["id", 156101], ["id", 156204], ["id", 156267], ["id", 15643
0], ["id", 156487], ["id", 156636], ["id", 156835], ["id", 157212], ["id", 157462], ["id", 157567], ["id", 157568], ["id", 157569], ["id", 157915], ["id", 158217], ["id", 159435], ["id", 159443], ["id", 159528], [
"id", 159536], ["id", 159583], ["id", 159596], ["id", 159777], ["id", 160153], ["id", 160172], ["id", 160220], ["id", 160248], ["id", 160258], ["id", 160259], ["id", 160260], ["id", 160261], ["id", 160263], ["id",
 160264], ["id", 160265], ["id", 160266], ["id", 160267], ["id", 160271], ["id", 160272], ["id", 160273], ["id", 160274], ["id", 160275], ["id", 160277], ["id", 160278], ["id", 160279], ["id", 160280], ["id", 1602
81], ["id", 160282], ["id", 160283], ["id", 160284], ["id", 160285], ["id", 160286], ["id", 160287], ["id", 160288], ["id", 160289], ["id", 160290], ["id", 160291], ["id", 160292], ["id", 160293], ["id", 160294],
["id", 160295], ["id", 160296]]
>>> counting for mastodon observed=1.0 expected=1.0
>>> counting for bonjour observed=1.0 expected=1.0
>>> counting for gamedev observed=1.0 expected=1.0
>>> counting for indiedev observed=1.0 expected=1.0
>>> counting for opensource observed=1.0 expected=2.0
>>> counting for fediverse observed=1.0 expected=3.0
>>> counting for patreon observed=1.0 expected=1.0
>>> counting for np observed=1.0 expected=2.0
>>> counting for communs observed=2.0 expected=1.0
>>> counting for android observed=2.0 expected=4.0
>>> counting for video observed=1.0 expected=2.0
[...]

Donc il parcourt bien les tags, et il n'y en a pas de trending (le seuil est à 5 utilisations le même jour). Et effectivement la liste est vide (ça a au passage reset les vieux).

Reste à comprendre pourquoi cette tâche n'était pas exécutée correctement jusque là. Et je vais aussi essayer de déclencher un trending avec un compte de test.

Correction avec plusieurs comptes de test. Il faut plusieurs utilisateurs. Je vais demander à tout le monde.

Confirmé : la routine fonctionne bien et met bien à jour. Je vais tenter de comprendre pourquoi elle n'est pas exécutée par Sidekiq.

Les tâches sont à nouveau lancées après avoir flush le redis.

added Progression / Fait label and removed Progression / En cours label

Pour moi c'est résolu, on va voir si les trendings apparaissent dans l'aprèm

marked the checklist item Les tendances ne se mettent pas à jour as completed

Ca semble en effet réglé pour les tendances mais pas pour le second problème.

added Progression / En cours label and removed Progression / Fait label

Qu'est-ce que tu utilises pour vérifier le second problème ? Si c'est un vieux message pour lequel les hashtags n'auraient pas été indexés, malheureusement il ne repassera pas dessus. Il faut tester avec des messages récents. Pour ma part j'ai l'impression que c'est ok.

Par exemple le tag #baffie2022 dont le dernier message date d'il y a 3 jours :

unassigned @Angedestenebres

Pour le deuxième souci, le tag qui posait problème fonctionne normalement. Par contre, impossible de l'ajouter dans mon profil mais je vois que j'ai 10 tags et cela pourrait venir d'une limitation.

Il n'y a plus de réel souci ici. Pour les hashtags que l'on ne peut pas rajouter dans son profil, c'est parce que c'est limité à 10. En soi, c'est juste l'erreur affichée qui n'est pas claire :

Il n'y a aucune autre précision x)

Ce problème est répertorié ici : https://github.com/tootsuite/mastodon/issues/14610

marked the checklist item Certains hashtags ne peuvent pas être ajoutés à son profil et ne remonte pas en publique alors que les toots sont publiques. as completed

closed

Divers problèmes avec les hashtags

Designs

Child items ...

Activity