View Issue Details

This bug affects 2 person(s).
 16
IDProjectCategoryView StatusLast Update
17747Feature requestsRemoteControlpublic2023-10-16 07:18
Reportertbart Assigned Togalads  
PrioritynoneSeverityfeature 
Status acknowledgedResolutionopen 
Summary17747: Add export_survey_archive/export_survey_structure to API (working implementation attached)
Description

I am doing pretty complex automation stuff with limesurvey (some more details/tools might follow later if people are interested), one of them being the creation of surveys based on survey templates (and manipulating their XML in between), the other one being copying surveys to a given ID/changed name (as copy_survey does to allow to set a target ID and import_survey is buggy, see https://bugs.limesurvey.org/view.php?id=17746).

As I needed an export of lss/lsa anyway, I fixed both issues by implementing the two functions and manipulating the name and ID in the XML, then reimporting the surveys.

However, I am not a developer (probably one of the functional/procedural type, but definitely no OO and php dev) but just a sysadmin with automation skills.

There's a good amount of code duplication, which I know is ugly. The addtoZip() implementation also does not seem right like I did it. However, I had to work with what's available.
And I can say for sure that this (or the concept at least) is tried and tested for over a year and works flawlessly.

If you think this might be useful to others, please consider implementing it along the structure outlined in the attached diff (but please correct the ugliness I produced :-) )
Maybe even more deduplication could be performed by using a common function for the fetching part in these stated export functions here and copy_survey as they do the same anyway.

Feel free to use the code as you see fit, no copyright/strings attached.

Thanks a lot!

PS: I hit the limit of bash's max. commandline length when trying to import an lss as a string. I solved this by eventually creating an lsa from it by simply zipping it though this is not documented like this. I think it would be practical to either state this possibility in the documentation or probably make lss a zipped format, though I think just documenting it in the API documentation (that's the place devs go to to find how they can perform API tasks) might be enough without breaking established formats.
I don't know if this actually misuses the lsa import or this is legit. I just wanted you to know there's a need for this and there's also a solution/workaround at the moment that should be kept, as it works and has practical relevance.

TagsNo tags attached.
Attached Files
api_export_survey.patch (7,413 bytes)   
--- application/helpers/remotecontrol/remotecontrol_handle.php.org	2020-02-13 16:39:43.137157937 +0100
+++ application/helpers/remotecontrol/remotecontrol_handle.php	2020-05-26 17:19:04.892983689 +0200
@@ -255,6 +255,146 @@
     }
 
     /**
+     * RPC Routine to export a survey archive (LSA).
+     *
+     * @access public
+     * @param string $sSessionKey Auth credentials
+     * @param int $iSurveyID_org Id of the survey
+     * @return string|array in case of success : Base64 encoded string of the .lsa file. On failure array with error information.
+     * */
+    public function export_survey_archive ($sSessionKey, $iSurveyID_org)
+    {
+        $iSurveyID = (int) $iSurveyID_org;
+        if (!$this->_checkSessionKey($sSessionKey)) {
+            return array('status' => 'Invalid session key');
+        }
+        $aData['bFailed'] = false; // Put a var for continue
+        if (!$iSurveyID) {
+            $aData['sErrorMessage'] = "No survey ID has been provided. Cannot export survey";
+            $aData['bFailed'] = true;
+        } elseif (!Survey::model()->findByPk($iSurveyID)) {
+            $aData['sErrorMessage'] = "Invalid survey ID";
+            $aData['bFailed'] = true;
+        } elseif (!Permission::model()->hasSurveyPermission($iSurveyID, 'surveycontent', 'export') && !Permission::model()->hasSurveyPermission($iSurveyID, 'surveycontent', 'export')) {
+            $aData['sErrorMessage'] = "You don't have sufficient permissions.";
+            $aData['bFailed'] = true;
+        } else {
+            $aExcludes = array();
+            $aExcludes['dates'] = true;
+            $btranslinksfields = true;
+            Yii::app()->loadHelper('export');
+
+            /* START CODE DUPLICATION
+             * taken from application/controllers/admin/export.php:_exportarchive
+             * This is ugly.
+             * Sorry, I am just a sysadmin, no PHP dev
+             * The mentioned function would somehow need to be public to actually use it
+             * However, I barely understand OO programming
+             *
+             * I also added addToZip to export_helper.php and made it non-private
+             * to be able to use it here. Ugly as well.
+             *
+             * Still, it works, so the concept should be clear.
+             */
+            $survey = Survey::model()->findByPk($iSurveyID);
+
+            // $aSurveyInfo = getSurveyInfo($iSurveyID); // unused, even in export.php
+
+            $sTempDir = Yii::app()->getConfig("tempdir");
+
+            $aZIPFileName = $sTempDir.DIRECTORY_SEPARATOR.randomChars(30);
+            $sLSSFileName = $sTempDir.DIRECTORY_SEPARATOR.randomChars(30);
+            $sLSRFileName = $sTempDir.DIRECTORY_SEPARATOR.randomChars(30);
+            $sLSTFileName = $sTempDir.DIRECTORY_SEPARATOR.randomChars(30);
+            $sLSIFileName = $sTempDir.DIRECTORY_SEPARATOR.randomChars(30);
+
+            Yii::import('application.libraries.admin.pclzip', true);
+            $zip = new PclZip($aZIPFileName);
+
+            file_put_contents($sLSSFileName, surveyGetXMLData($iSurveyID));
+
+            addToZip($zip, $sLSSFileName, 'survey_'.$iSurveyID.'.lss');
+
+            unlink($sLSSFileName);
+
+            if ($survey->isActive) {
+                getXMLDataSingleTable($iSurveyID, 'survey_'.$iSurveyID, 'Responses', 'responses', $sLSRFileName, false);
+                addToZip($zip, $sLSRFileName, 'survey_'.$iSurveyID.'_responses.lsr');
+                unlink($sLSRFileName); 
+            }   
+
+            if ($survey->hasTokensTable) {
+                getXMLDataSingleTable($iSurveyID, 'tokens_'.$iSurveyID, 'Tokens', 'tokens', $sLSTFileName);
+                addToZip($zip, $sLSTFileName, 'survey_'.$iSurveyID.'_tokens.lst');
+                unlink($sLSTFileName); 
+            }   
+
+            if (isset($survey->hasTimingsTable) && $survey->hasTimingsTable == 'Y') {
+                getXMLDataSingleTable($iSurveyID, 'survey_'.$iSurveyID.'_timings', 'Timings', 'timings', $sLSIFileName);
+                addToZip($zip, $sLSIFileName, 'survey_'.$iSurveyID.'_timings.lsi');
+                unlink($sLSIFileName); 
+            }   
+
+            if (is_file($aZIPFileName)) {
+            /* END CODE DUPLICATION */
+
+                    $sResult = file_get_contents($aZIPFileName);
+					unlink($aZIPFileName);
+                } else {
+                    $aData['bFailed'] = true;
+                    $aData['sErrorMessage'] = 'Error creating lsa archive';
+                }
+        }
+        if ($aData['bFailed']) {
+            return array('status' => 'Export failed', 'error'=> $aData['sErrorMessage']);
+        } else {
+            return base64_encode($sResult);
+        }
+    }
+    /**
+     * RPC Routine to export a survey structure (LSS).
+     *
+     * @access public
+     * @param string $sSessionKey Auth credentials
+     * @param int $iSurveyID_org Id of the survey
+     * @return string|array in case of success : Base64 encoded string of the .lss file. On failure array with error information.
+     * */
+    public function export_survey_structure ($sSessionKey, $iSurveyID_org)
+    {
+        $iSurveyID = (int) $iSurveyID_org;
+        if (!$this->_checkSessionKey($sSessionKey)) {
+            return array('status' => 'Invalid session key');
+        }
+        $aData['bFailed'] = false; // Put a var for continue
+        if (!$iSurveyID) {
+            $aData['sErrorMessage'] = "No survey ID has been provided. Cannot export survey";
+            $aData['bFailed'] = true;
+        } elseif (!Survey::model()->findByPk($iSurveyID)) {
+            $aData['sErrorMessage'] = "Invalid survey ID";
+            $aData['bFailed'] = true;
+        } elseif (!Permission::model()->hasSurveyPermission($iSurveyID, 'surveycontent', 'export') && !Permission::model()->hasSurveyPermission($iSurveyID, 'surveycontent', 'export')) {
+            $aData['sErrorMessage'] = "You don't have sufficient permissions.";
+            $aData['bFailed'] = true;
+        } else {
+            $aExcludes = array();
+            $aExcludes['dates'] = true;
+            $btranslinksfields = true;
+            Yii::app()->loadHelper('export');
+            $exportsurveystructuredata = surveyGetXMLData($iSurveyID, $aExcludes);
+            if ($exportsurveystructuredata) {
+                $sResult = $exportsurveystructuredata;
+            } else {
+                $aData['bFailed'] = true;
+            }
+        }
+        if ($aData['bFailed']) {
+            return array('status' => 'Export failed', 'error'=> $aData['sErrorMessage']);
+        } else {
+            return base64_encode($sResult);
+        }
+    }
+
+    /**
      * Copy survey (RPC function)
      *
      * @access public
--- application/helpers/export_helper.php.org	2020-05-26 17:22:50.808991768 +0200
+++ application/helpers/export_helper.php	2020-05-26 17:07:19.924958477 +0200
@@ -13,6 +13,26 @@
 */
 
 /**
+ * CODE DUPLICATION! This is also part of export.php
+ * Sorry I can't solve this any better...
+ * @param PclZip $zip
+ * @param string $name
+ * @param string $full_name
+ */
+
+function addToZip($zip, $name, $full_name)
+{
+    $zip->add(
+    array(
+    array(
+    PCLZIP_ATT_FILE_NAME => $name,
+    PCLZIP_ATT_FILE_NEW_FULL_NAME => $full_name
+    )
+    )
+    );
+}
+
+/**
 * Strips html tags and replaces new lines
 *
 * @param $string
api_export_survey.patch (7,413 bytes)   
Bug heat16
Story point estimate
Users affected %

Users monitoring this issue

Amaury VAN ESPEN

Activities

galads

galads

2022-02-01 13:43

reporter   ~68164

Hello,

Thank you for creating a patch. Can you create a PR?

tbart

tbart

2022-02-03 21:56

reporter   ~68201

Basically I use git everyday with my local instance but have never used it for shared projects. My team is currently extremely overloaded so yes, maybe, but this can take pretty long as I'd have to read up on how to do this and find a free slot once we're better staffed again.

stefheiselzzz

stefheiselzzz

2023-07-28 10:52

viewer   ~76330

Hey,

As a fellow sysadmin with automation skills, I can appreciate the complexity of the automation stuff the author is dealing with. The struggle with code duplication and the feeling that the implementation might not be perfect is something many of us face in our non-developer roles. But hey, we do what we can with the resources available, right?

The author's approach of implementing the export_survey_archive/export_survey_structure functions and manipulating the XML sounds brilliant to me. Sometimes, the best solutions come from thinking outside the box. And kudos for sharing the code without any strings attached! That's the spirit of open collaboration.

In terms of advice, I would suggest exploring the option of creating a common function for the fetching part in the export functions and copy_survey. This could potentially help with deduplication and improve code maintenance. Another question is, do you own this technology: https://andersenlab.com/services/devops ? It helped me in some issues, just knowing the theory and how it works. Try it!

Regarding the limitation of bash's max commandline length, the workaround of creating an lsa by zipping the lss is quite clever. It's true that documenting this possibility in the API documentation would be beneficial for other developers facing the same issue.

All in all, I'm really impressed by the resourcefulness and problem-solving skills displayed here. Keep up the good work, and I'm sure your contributions will be valuable to the Limesurvey community.

Amaury VAN ESPEN

Amaury VAN ESPEN

2023-10-16 07:18

reporter   ~77713

Hello,
I've added the pull request with the patch on github.
see https://github.com/LimeSurvey/LimeSurvey/pull/3538
regards
Amaury

Issue History

Date Modified Username Field Change
2021-11-22 10:21 tbart New Issue
2021-11-22 10:21 tbart File Added: api_export_survey.patch
2022-02-01 13:43 galads Note Added: 68164
2022-02-01 13:43 galads Bug heat 0 => 2
2022-02-01 13:43 galads Assigned To => galads
2022-02-01 13:43 galads Status new => acknowledged
2022-02-03 21:56 tbart Note Added: 68201
2022-02-03 21:56 tbart Bug heat 2 => 4
2023-07-28 10:43 stefheiselzzz Bug heat 4 => 6
2023-07-28 10:52 stefheiselzzz Note Added: 76330
2023-10-16 06:56 Amaury VAN ESPEN Issue Monitored: Amaury VAN ESPEN
2023-10-16 06:56 Amaury VAN ESPEN Bug heat 6 => 8
2023-10-16 06:59 Amaury VAN ESPEN Bug heat 8 => 14
2023-10-16 07:18 Amaury VAN ESPEN Note Added: 77713
2023-10-16 07:18 Amaury VAN ESPEN Bug heat 14 => 16