View Issue Details

This bug affects 3 person(s).
 20
IDProjectCategoryView StatusLast Update
19371Bug reportsOtherpublic2024-04-16 13:51
Reporterc_schmitz Assigned Tolajosarpad  
PrioritynoneSeveritypartial_block 
Status resolvedResolutionfixed 
Product Version6.4.x 
Summary19371: Survey list page extremely slow when having alot of surveys and inheritance is used for 'List publicly'
Description

We have a database with 6000 surveys spread to about 10 different survey groups.
All survey groups have their "list public" property set to Inherited, except for the default one.

When opening the index page, even though not a single survey is actually listed, it takes about 10-20 second to load the index page.
If you set the 'List public' property of all surveys to 'No', the index page takes 2-3 seconds (which still is kinda slow but more acceptable)

Steps To Reproduce

Steps to reproduce

Create 6000 active surveys, 10 survey groups, assign 600 surveys to each survey group , set 'List survey public' to inherit in all surveys and survey groups (except the default one, where it is set to 'no')
Open the public index page.

Expected result

Page should load immediately, no surveys listed.

Actual result

Page needs 10-20 seconds, no surveys listed.

(Write here what happened instead)

TagsNo tags attached.
Bug heat20
Complete LimeSurvey version number (& build)6.4.1+240108
I will donate to the project if issue is resolvedNo
Browser
Database type & version.
Server OS (if known)
Webserver software & version (if known)
PHP Version.

Relationships

related to 18473 closedDenisChenu Bug reports Survey listing : multiple call of template 
related to 19426 closedDenisChenu Bug reports Survey listing : multiple call of user table 

Users monitoring this issue

JanE

Activities

DenisChenu

DenisChenu

2024-01-22 10:07

developer   ~79261

But : showing 6000 survey take more times than showing 10 :)

2-3 seconds : maybe need a static var for amll;groups

Related issue : https://bugs.limesurvey.org/view.php?id=18473

Can you activate debugsql and check number of requests ?

c_schmitz

c_schmitz

2024-01-30 17:42

administrator   ~79359

There are no surveys shown at all. The default survey groups has set 'List public' set to false.
Still it is so slow due to inheritance.

DenisChenu

DenisChenu

2024-01-31 07:58

developer   ~79362

Last edited: 2024-01-31 07:59

debugsql show (have 36 survey activated)

  • 20 times User (have o nly one user) system.db.CDbCommand.query(SELECT * FROM lime_users t WHERE t.uid=1 LIMIT 1)
  • 16 times X 9template : system.db.CDbCommand.query(SELECT template.id AS t1_c0, template.name AS t1_c1, template.folder AS t1_c2, template.title AS t1_c3, template.creation_date AS t1_c4, template.author AS t1_c5, template.author_email AS t1_c6, template.author_url AS t1_c7, template.copyright AS t1_c8, template.license AS t1_c9, template.version AS t1_c10, template.api_version AS t1_c11, template.view_folder AS t1_c12, template.files_folder AS t1_c13, template.description AS t1_c14, template.last_update AS t1_c15, template.owner_id AS t1_c16, template.extends AS t1_c17 FROM lime_templates template WHERE (template.name=:ypl0). Bound with :ypl0='bootswatch')
  • 10 times Survey group (default) system.db.CDbCommand.query(SELECT t.gsid AS t0_c0, t.owner_id AS t0_c1, t.admin AS t0_c2, t.adminemail AS t0_c3, t.anonymized AS t0_c4, t.format AS t0_c5, t.savetimings AS t0_c6, t.template AS t0_c7, t.datestamp AS t0_c8, t.usecookie AS t0_c9, t.allowregister AS t0_c10, t.allowsave AS t0_c11, t.autonumber_start AS t0_c12, t.autoredirect AS t0_c13, t.allowprev AS t0_c14, t.printanswers AS t0_c15, t.ipaddr AS t0_c16, t.refurl AS t0_c17, t.showsurveypolicynotice AS t0_c18, t.publicstatistics AS t0_c19, t.publicgraphs AS t0_c20, t.listpublic AS t0_c21, t.htmlemail AS t0_c22, t.sendconfirmation AS t0_c23, t.tokenanswerspersistence AS t0_c24, t.assessments AS t0_c25, t.usecaptcha AS t0_c26, t.bounce_email AS t0_c27, t.attributedescriptions AS t0_c28, t.emailresponseto AS t0_c29, t.emailnotificationto AS t0_c30, t.tokenlength AS t0_c31, t.showxquestions AS t0_c32, t.showgroupinfo AS t0_c33, t.shownoanswer AS t0_c34, t.showqnumcode AS t0_c35, t.showwelcome AS t0_c36, t.showprogress AS t0_c37, t.questionindex AS t0_c38, t.navigationdelay AS t0_c39, t.nokeyboard AS t0_c40, t.alloweditaftercompletion AS t0_c41, t.ipanonymize AS t0_c42, SurveysGroups.gsid AS t1_c0, SurveysGroups.name AS t1_c1, SurveysGroups.title AS t1_c2, SurveysGroups.template AS t1_c3, SurveysGroups.description AS t1_c4, SurveysGroups.sortorder AS t1_c5, SurveysGroups.owner_id AS t1_c6, SurveysGroups.parent_id AS t1_c7, SurveysGroups.created AS t1_c8, SurveysGroups.modified AS t1_c9, SurveysGroups.created_by AS t1_c10, SurveysGroups.alwaysavailable AS t1_c11 FROM lime_surveys_groupsettings t LEFT OUTER JOIN lime_surveys_groups SurveysGroups ON (SurveysGroups.gsid=t.gsid) WHERE (t.gsid=1))
  • 10 times system.db.CDbCommand.query(SELECT * FROM lime_surveys_groupsettings t WHERE t.gsid=0 LIMIT 1)

My opinion :
Need optimisation (static or better : caching) for template and SurveyGroups option

template-call.png (382,446 bytes)
c_schmitz

c_schmitz

2024-02-02 16:35

administrator   ~79396

I totally agree.

When looking at the survey list page with only 1 survey listed I see 61 queries being run. Most of the repeated ones are for the template system. I think with some work the number of queries can be reduced by 30-50%.

DenisChenu

DenisChenu

2024-02-14 17:25

developer   ~79497

This one : https://github.com/LimeSurvey/LimeSurvey/pull/3737
For SurveyGroupSettings : only DB call

DenisChenu

DenisChenu

2024-02-14 17:43

developer   ~79498

For template : seems we don't( use (when need it) TemplateConfiguration::getInstanceFromTemplateName

DenisChenu

DenisChenu

2024-02-14 18:02

developer   ~79499

For template : https://github.com/LimeSurvey/LimeSurvey/pull/3738

DenisChenu

DenisChenu

2024-02-15 16:44

developer   ~79525

OK : SurveyGroup broken : i use the other for template

DenisChenu

DenisChenu

2024-02-15 18:40

developer   ~79530

master : group settings : https://github.com/LimeSurvey/LimeSurvey/pull/3737

DenisChenu

DenisChenu

2024-02-15 18:41

developer   ~79531

This one can be moved to cache for develop.
I think we only use model->save for SureyGroupSettings update ?
Maybe some saveAll ?

DenisChenu

DenisChenu

2024-02-16 10:42

developer   ~79533

I start with instance with options (inherited from parent) … but need more rewrite.
Since survey update instance of SurveyGroup (WTF ?)

DenisChenu

DenisChenu

2024-02-26 09:36

developer   ~79624

For 5X

tibor.pacalat

tibor.pacalat

2024-03-05 17:24

administrator   ~79728

https://github.com/LimeSurvey/LimeSurvey/pull/3738 has been merged in, but the number was wrong (18473), so it is not shown here.

@DenisChenu do we need this fix for 5.x as well?

DenisChenu

DenisChenu

2024-03-05 17:29

developer   ~79731

do we need this fix for 5.x as well?

In my opinion : yes , same for the 2 others.

Here it's more : https://github.com/LimeSurvey/LimeSurvey/pull/3737 (SurveyGroups)

DenisChenu

DenisChenu

2024-03-05 17:30

developer   ~79732

OK, confirm needed for 5.X too :)

I do the 3 for 5 (separate)

tibor.pacalat

tibor.pacalat

2024-03-05 17:31

administrator   ~79733

Ok, then please make a PR for 5.x when you get some time.

DenisChenu

DenisChenu

2024-03-07 12:31

developer   ~79756

5.X : https://github.com/LimeSurvey/LimeSurvey/pull/3787

Issue History

Date Modified Username Field Change
2024-01-22 10:02 c_schmitz New Issue
2024-01-22 10:03 c_schmitz Steps to Reproduce Updated
2024-01-22 10:07 DenisChenu Note Added: 79261
2024-01-22 10:07 DenisChenu Bug heat 0 => 2
2024-01-22 10:19 c_schmitz Steps to Reproduce Updated
2024-01-29 09:17 Mazi Bug heat 2 => 8
2024-01-30 17:42 c_schmitz Note Added: 79359
2024-01-30 17:42 c_schmitz Bug heat 8 => 10
2024-01-31 07:58 DenisChenu Note Added: 79362
2024-01-31 07:58 DenisChenu File Added: template-call.png
2024-01-31 07:59 DenisChenu Note Edited: 79362
2024-01-31 07:59 DenisChenu Relationship added related to 18473
2024-02-02 16:35 c_schmitz Note Added: 79396
2024-02-14 17:25 DenisChenu Note Added: 79497
2024-02-14 17:43 DenisChenu Note Added: 79498
2024-02-14 17:53 DenisChenu Assigned To => DenisChenu
2024-02-14 17:53 DenisChenu Status new => assigned
2024-02-14 18:02 DenisChenu Note Added: 79499
2024-02-15 16:44 DenisChenu Relationship added related to 19426
2024-02-15 16:44 DenisChenu Note Added: 79525
2024-02-15 18:40 DenisChenu Status assigned => ready for code review
2024-02-15 18:40 DenisChenu Note Added: 79530
2024-02-15 18:41 DenisChenu Note Added: 79531
2024-02-16 10:41 DenisChenu Assigned To DenisChenu => c_schmitz
2024-02-16 10:42 DenisChenu Note Added: 79533
2024-02-26 09:36 DenisChenu Status ready for code review => assigned
2024-02-26 09:36 DenisChenu Note Added: 79624
2024-03-05 17:24 tibor.pacalat Note Added: 79728
2024-03-05 17:24 tibor.pacalat Bug heat 10 => 12
2024-03-05 17:24 tibor.pacalat Assigned To c_schmitz => DenisChenu
2024-03-05 17:29 DenisChenu Note Added: 79731
2024-03-05 17:30 DenisChenu Note Added: 79732
2024-03-05 17:31 tibor.pacalat Note Added: 79733
2024-03-07 12:31 DenisChenu Status assigned => ready for code review
2024-03-07 12:31 DenisChenu Note Added: 79756
2024-03-07 12:32 DenisChenu Assigned To DenisChenu => lajosarpad
2024-03-23 11:17 JanE Issue Monitored: JanE
2024-03-23 11:17 JanE Bug heat 12 => 14
2024-04-16 13:51 tibor.pacalat Status ready for code review => resolved
2024-04-16 13:51 tibor.pacalat Resolution open => fixed
2024-04-16 14:23 JanE Bug heat 14 => 20